NASA/CR-2001-2 10845 



The Effect of Ownship Information 
and NexRad Resolution on Pilot 
Decision Making in the Use of a Cockpit 
Weather Information Display 


Paul F. Novacek, Malcolm A. Burgess, Michael L. Heck, and Alan F. Stokes 
RTI International, Hampton, Virginia 


December 2001 



The NASA STI Program Office ... in Profile 


Since its founding, NASA has been dedicated to 
the advancement of aeronautics and space 
science. The NASA Scientific and Technical 
Information (STI) Program Office plays a key 
part in helping NASA maintain this important 
role. 

The NASA STI Program Office is operated by 
Langley Research Center, the lead center for 
NASA’s scientific and technical information. The 
NASA STI Program Office provides access to the 
NASA STI Database, the largest collection of 
aeronautical and space science STI in the world. 
The Program Office is also NASA’s institutional 
mechanism for disseminating the results of its 
research and development activities. These 
results are published by NASA in the NASA STI 
Report Series, which includes the following 
report types: 

• TECHNICAL PUBLICATION. Reports of 
completed research or a major significant 
phase of research that present the results of 
NASA programs and include extensive 
data or theoretical analysis. Includes 
compilations of significant scientific and 
technical data and information deemed to 
be of continuing reference value. NASA 
counterpart of peer-reviewed formal 
professional papers, but having less 
stringent limitations on manuscript length 
and extent of graphic presentations. 

• TECHNICAL MEMORANDUM. Scientific 
and technical findings that are preliminary 
or of specialized interest, e.g., quick release 
reports, working papers, and 
bibliographies that contain minimal 
annotation. Does not contain extensive 
analysis. 

• CONTRACTOR REPORT. Scientific and 
technical findings by NASA-sponsored 
contractors and grantees. 


• CONFERENCE PUBLICATION. Collected 
papers from scientific and technical 
conferences, symposia, seminars, or other 
meetings sponsored or co-sponsored by 
NASA. 

• SPECIAL PUBLICATION. Scientific, 
technical, or historical information from 
NASA programs, projects, and missions, 
often concerned with subjects having 
substantial public interest. 

• TECHNICAL TRANSLATION. English- 
language translations of foreign scientific 
and technical material pertinent to NASA’s 
mission. 

Specialized services that complement the STI 
Program Office’s diverse offerings include 
creating custom thesauri, building customized 
databases, organizing and publishing research 
results ... even providing videos. 

For more information about the NASA STI 
Program Office, see the following: 

• Access the NASA STI Program Home Page 

at http://www.sti.nasa.gov 

• E-mail your question via the Internet to 
help@sti.nasa.gov 

• Fax your question to the NASA STI Help 
Desk at (301) 621-0134 

• Phone the NASA STI Help Desk at 
(301) 621-0390 

• Write to: 

NASA STI Help Desk 

NASA Center for AeroSpace Information 

7121 Standard Drive 

Hanover, MD 21076-1320 



NAS A/CR-200 1-21 0845 



The Effect of Ownship Information 
and NexRad Resolution on Pilot Decision 
Making in the Use of a Cockpit Weather 
Information Display 


Paul F. Novacek, Malcolm A. Burgess, Michael L. Heck, and Alan F. Stokes 
RTI International, Hampton, Virginia 


National Aeronautics and 
Space Administration 

Langley Research Center Prepared for Langley Research Center 

Hampton, Virginia 23681-2199 under Contract NAS1-99074 


December 2001 



Acknowledgment 


The authors wish to acknowledge the help of several colleagues whose contributions 
were key to the successful completion of this experiment. Mr. James Henion from RTI 
International prepared specialized software required for this simulation and operated the 
simulation. Mr. James Murray, Air Traffic Control consultant, developed and executed 
the ATC scripts used during the simulation runs. Dr. Raymon McAdaragh of the Federal 
Aviation Administration obtained and organized the weather documentation used by the 
subject pilots in the experiment. Dr. Holly Travers, of Rensselaer Polytechnic Institute, 
performed the statistical analyses of the data obtained in the experiment. Mr. Alex Viale, 
Masters Student at Rensselaer Polytechnic Institute, assisted in the extensive literature 
search. 

We would like to thank Mr. Brian Haynes of Honeywell for cooperation in the modification 
and use of a prototype Flight Information Services Data Link display system in the 
simulation. We would especially like to acknowledge the contributions of the pilots. They 
all volunteered their time and traveling expenses. Some Raveled to Hampton, Virginia 
from as far away as Connecticut. These individuals were magnificent in the generosity of 
their time, patience and piloting skills. 


Available from: 


NASA Center for AeroSpace Information (CASI) 
7121 Standard Drive 
Hanover, MD 21076-1320 
(301) 621-0390 


National Technical Information Service (NTIS) 
5285 Port Royal Road 
Springfield, VA 22161-2171 
(703) 605-6000 




Table of Contents 


Executive Summary vii 

1 Introduction 1 

1.1 Potential Issues with Datalinked Weather Displays 2 

1 .2 Survey of Relevant Literature 3 

2 Participants 5 

2.1 FAA Data Link Office 5 

2.2 NASA AWIN Project 5 

2.3 RTI International 5 

3 Methodology 7 

3.1 Experiment Design 7 

3.2 Pilot Selection Process 9 

3.3 Experimental Apparatus 10 

3.3. 1 Pre-Flight Planning T ools 10 

3.3.2 Flight Simulation Facility 10 

3.3.3 Cockpit Weather Information Display 13 

4 Procedure 17 

4. 1 Key Phases of Experiment Procedure 17 

4.1.1 Experiment Briefing 17 

4. 1 .2 Simulator Familiarization 17 

4.1.3 Pre-Flight Planning 17 

4.1.4 Simulator Mission 18 

4.1.5 Post-mission De-briefing 19 

4.2 Flight Procedures 19 

4.2.1 Mission Scenario 19 

4.2.2 First Leg of Flight - Newport News to Richmond 21 

4.2.3 Second Leg of Flight - Richmond to Wallops Island 23 

5 Results 25 

5.1 Subject Pilots 25 

5.2 Richmond and Wallops Island Decisions 26 

5.2.1 The Richmond Decision 26 

5.2.2 The Wallops Island Decision 28 

5.3 Results of Immediate Reaction Questionnaire 31 

5.4 Post-flight Weather Display Questionnaire 35 

6 Quantitative and Qualitative Assessments 41 

6.1 Subject Group Comparisons 41 

6.2 Quantitative Assessments 41 

6.2.1 Effect of Ownship Position on Pilots’ Decisions 42 

6.2.2 Effect of NEXRAD Image Cell Size on Pilot Decisions 44 

6.2.3 Rationale for Pilots’ Decisions 47 

6.2.4 Combination of Decisions from Previous and Current Experiments . 48 

6.2.5 Risk Aversion, Weather Knowledge and Experience Analysis 50 



6.2.5. 1 Richmond Hierarchical Regression Analysis 50 

6. 2. 5. 2 Wallops Island Hierarchical Regression Analysis 51 

6. 2. 5. 3 Combined Hierarchical Regression Analysis 52 

6.3 Qualitative Assessments 52 

6.3.1 Workload Issues 53 

6. 3. 1.1 Overall Mission Workload 53 

6. 3. 1.2 Workload in use of Weather Display 55 

6.3.2 Use of Available Weather Data Sources 56 

6.3.2. 1 Richmond Area Weather Sources 57 

6. 3.2. 2 Wallops Island Weather Sources 59 

6.3.3 Interpretation of the Weather Information Display 60 

6.3.3. 1 Judging Proximity to Hazardous Weather 61 

6. 3.3. 2 Situational Awareness 61 

6. 3.3. 3 Recognizing and Interpreting Effects of Delay 62 

6. 3.3.4 NEXRAD Image Cell Size Effects on Richmond Decision 62 

6.3.4 Retrospective Pilot Comments 63 

7 Conclusions 67 

7.1 Weather Information Display Interpretation Issues 68 

7.1.1 Position Determination 68 

7.1.2 NEXRAD Image Cell Resolution 68 

7.1.3 Workload Reduction 69 

7.2 Weather Source Information Issues 69 

7.3 METAR Issues 69 

7.3.1 METAR Coding 69 

7.3.2 METAR Latency 70 

7.3.3 Graphical METARs 70 

7.4 Judging Proximity and Movement of Hazardous Weather 70 

7.5 Stimulus Area Effect 70 

8 Recommendations 71 

8.1 AIM and Advisory Circular Recommendations 71 

8.2 Recommendations for Weather Display Manufacturers 72 

8.2.1 Provide Ownship Position 72 

8.2.2 Provide Direction and Rate of Weather Motion 72 

8.2.3 Provide for Intuitive Distance Determination 72 

8.2.4 Provide for Intuitive NEXRAD Image Age 73 

8.2.5 Provide METAR Code Translation 73 

9 Recommendations for Further Research 75 

9.1 Conduct Evaluations of Cell Movement Depictions 75 

9.2 Develop Concepts for Display of Predictive Weather Products 75 

9.3 Determine Optimum NEXRAD Image Cell Resolution 75 

9.4 Develop Training for Weather Information Displays 75 

10 Bibliography 77 

Appendix A. Flight Information Services Description 81 

Appendix B. Aeronautical Information Manual FISDL Guidance 83 

Appendix C. The Risk Assessment Task (RAT) 87 

Appendix D. Weather Knowledge Questionnaire and Key 91 


IV 



Appendix E. Pre-Flight Weather Briefing 97 

Appendix F. Cockpit Research Facility Description 101 

Appendix G. NEXRAD Mosaic Images - 4km Cells Ill 

Appendix H. NEXRAD Mosaic Images - 8km Cells 117 

Appendix I. Experiment Briefing 123 

Appendix J. Simulator Briefing and Training 133 

Appendix K. Observer Form 135 

Appendix L. Immediate Reactions Questionnaire 137 

Appendix M. Structured Interview Guide 141 

Appendix N. Weather Display Questionnaire 145 

Appendix O. Air Traffic Control Scripts 149 

Appendix P. Enroute Weather Report Scripts 155 


Tables 

T able 5-1 . Pilot Flight Hours 25 

Table 5-2. Decision Results 31 

Table 6-1. Overview of Richmond Decisions 46 

Table 6-2. Overview of Wallops Island Decisions 46 

Table 6-3. Richmond Regression Analysis Results 51 

Table 6-4. Wallops Island Regression Analysis Results 51 

Table 6-5. Combined Richmond and Wallops Island Regression Results 52 

Table F-1. Instruments and Indicators in the Instrument Panel 104 

Table F-2. Real-time Parameters Displayed During Operations 106 

Table F-3. Dictionary of Recordable Parameters and Inducible Faults 107 


Figures 

Figure 3-1. Relationship between previous and current experiment groups 8 

Figure 3-2. Experimental System Facility Layout 11 

Figure 3-3. Key Simulation Facility Stations 12 

Figure 3-4. Simulator Instrument Layout 13 

Figure 3-5. Weather Display Controls 14 

Figure 3-6. Weather Display Screen Labels 15 

Figure 3-7. NEXRAD Mosaic Image Precipitation Intensity Key 15 

Figure 3-8. Graphic METAR Key 16 

Figure 4-1. Flight plan route given to pilot 18 

Figure 4-2. Display Image When Approaching Richmond 21 

Figure 5-1. NEXRAD image at 1914Z 27 

Figure 5-2. NEXRAD image at 1921 Z 27 

Figure 5-3. NEXRAD image at 1935Z 29 


V 



Figure 6-1 . Effect of Ownship Display on Richmond Decision 43 

Figure 6-2. Effect of Ownship Display on Wallops Decision 43 

Figure 6-3. Richmond Decisions 44 

Figure 6-4. Wallops Island Decisions 45 

Figure 6-5. Richmond Decision Rationale 47 

Figure 6-6. Wallops Island Decision Rationale 48 

Figure 6-7. Combined Richmond Decisions 49 

Figure 6-8. Combined Wallops Island Decisions 50 

Figure 6-9. Pilots’ Perceived Workload for Entire Mission 54 

Figure 6-10. Percent of Flight Time Autopilot Used 54 

Figure 6-1 1 . Pilots’ Perception of Weather Display Workload 56 

Figure 6-12. Additional Weather Sources Used for the Richmond Decision 57 

Figure 6-13. Primary Source of Weather Information for Richmond Decision.... 58 

Figure 6-14. Additional Weather Sources for the Wallops Island Decision 59 

Figure 6-15. Primary Source of Weather for the Wallops Island Decision 60 

Figure 6-16. Comparison of Small versus Large NEXRAD Cells 63 

Figure F-1. Cockpit Research Facility 102 

Figure F-2. Instrument Panel, Controls and Indicators 103 

Figure F-3. Air Traffic Management Console Display 108 

Figure G-1 . 1900Z NEXRAD Mosaic Image — 4 km Cells 1 1 1 

Figure G-2. 1907Z NEXRAD Mosaic Image — 4 km Cells 1 1 1 

Figure G-3. 1914Z NEXRAD Mosaic Image — 4 km Cells 112 

Figure G-4. 1921Z NEXRAD Mosaic Image — 4 km Cells 112 

Figure G-5. 1928Z NEXRAD Mosaic Image — 4 km Cells 113 

Figure G-6. 1935Z NEXRAD Mosaic Image — 4 km Cells 113 

Figure G-7. 1942Z NEXRAD Mosaic Image — 4 km Cells 114 

Figure G-8. 1949Z NEXRAD Mosaic Image — 4 km Cells 114 

Figure G-9. 1956Z NEXRAD Mosaic Image — 4 km Cells 115 

Figure G-10. 2003Z NEXRAD Mosaic Image — 4 km Cells 115 

Figure H-1. 1900Z NEXRAD Mosaic Image — 8 km Cells 117 

Figure H-2. 1907Z NEXRAD Mosaic Image — 8 km Cells 117 

Figure H-3. 1914Z NEXRAD Mosaic Image — 8 km Cells 118 

Figure H-4. 1921Z NEXRAD Mosaic Image — 8 km Cells 118 

Figure H-5. 1928Z NEXRAD Mosaic Image — 8 km Cells 119 

Figure H-6. 1935Z NEXRAD Mosaic Image — 8 km Cells 119 

Figure H-7. 1942Z NEXRAD Mosaic Image — 8 km Cells 120 

Figure H-8. 1949Z NEXRAD Mosaic Image — 8 km Cells 120 

Figure H-9. 1956Z NEXRAD Mosaic Image — 8 km Cells 121 

Figure H-10. 2003Z NEXRAD Mosaic Image — 8 km Cells 121 


VI 



Executive Summary 

Reason for Experiment 


Significant progress has been made in recent years toward the provision of real-time data- 
linked weather information to pilots. One of the most recent efforts is the broadcast Flight 
Information Services Data-Link (FISDL) service — a government-industry partnership 
created to bring near real-time weather information into the cockpit. RTI International 
recently completed an experiment that explored the effects of a data-linked in-flight 
weather display on pilot decision making. In that previous experiment, two groups of pi- 
lots were placed in a full-mission simulator, 12 without the data-linked weather display 
and 13 with the display. The results found that although the weather display increased the 
pilot’s awareness of the weather situation, there were many issues that prevented the dis- 
play from being used to its full capability. 

This follow-on experiment explored specific issues that were uncovered in the previous 
experiment. The results will assist the manufacturers of weather display systems, and 
provide information for the FAA on certification and usability issues. 


Overview of Experiment 

The experiment described in this report explored the effects of adding ownship position 
symbology and changing the NEXRAD cell size resolution shown on the data-linked 
weather display. 

It is anticipated that with sufficient training, careful use of the cockpit weather display, 
and prudent pilot procedures in instrument flight conditions, the emerging cockpit 
weather display products will provide substantial improvements to the safety of flight. It 
is important to note that this experiment, like the one that preceded it, was designed spe- 
cifically to identify potential hazards in the use of cockpit weather displays. 

Every aspect of the design of this experiment was undertaken with this objective in mind, 
including subject pilot selection, subject pilot training, and the mission scenario. Pilots 
were selected so as to provide as wide and representative a range as possible of the ex- 
perience, knowledge of weather and risk aversion of the population of general aviation 
pilots who might use these emerging cockpit weather display products. The training pro- 
vided the subject pilots was carefully tailored so as to provide them with sufficient fa- 
miliarity with the experimental equipment to successfully accomplish the mission sce- 
nario, while at the same time creating a reasonable probability that within the population 
of subject pilots selected, potential hazards in the use of the equipment might become ap- 
parent. Likewise, the mission scenario incorporated in the experiment was selected to en- 
sure that it could be accomplished by the average pilot with careful attention by the sub- 
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ject pilot to the instrument flight procedures, but offered sufficient opportunity for obser- 
vation of human error in the use of the prototype cockpit weather display where such 
hazards might exist. 

The experiment was conducted with twenty-four current instrument rated pilots who were 
divided into two equal groups and presented with a challenging but realistic flight sce- 
nario involving weather containing significant embedded convective activity. All flights 
were flown in a full-mission simulation facility in simulated instrument meteorological 
conditions. The objective of the experiment was to investigate the possible misuse of the 
data-linked weather information with respect to the addition of ownship position symbol- 
ogy and a change in NEXRAD image cell resolution. The results of the experiment iden- 
tified the issues that need further investigation or consideration by the manufacturers and 
the FAA. 

The pilots were presented with a flight scenario that involved the delivery of critical 
medicine to the NASA-Wallops Flight Facility. Departing from the Newport News- 
Williamsburg International Airport (Virginia), the pilots were told that additional medi- 
cine was to be picked up at the Richmond International Airport enroute to Wallops. The 
two legs of the flight were designed to encounter convective weather in order to investi- 
gate both temporal and spatial issues related to the use of a weather information display. 

In addition to the weather display, all the normal weather information sources were also 
available. 

The 24 pilots were divided into two groups of 12 each. One group flew the mission with a 
weather display that used small NEXRAD image cells (4x4 km cells) and the second 
group was given a display that used large NEXRAD image cells (8x8 km cells). The 
weather display provided to both groups in this experiment included ownship position 
symbology, whereas in the previous experiment, ownship position information was not 
provided on the display. The small-cell (4x4 km) NEXRAD images were processed 
through a software filter that assigned the highest level radar return to the entire large 
cell. All aspects of the previous and current experiment were identical, except for the 
provision of ownship position and different NEXRAD image cell sizes. 

Both experiments were organized around two key decision points, a decision during an 
approach into the Richmond airport and a decision enroute to the Wallops Island airport. 
Both the Richmond and Wallops Island decisions were scored on a good/poor ordinal 
scale. At Richmond, a good decision was deemed to be one in which the pilot decided to 
abandon the approach prior to the Final Approach Fix, thus avoiding the hazardous 
weather by at least five nautical miles. A poor decision was deemed to be one in which 
the pilot continued with an approach for whatever reason, placing the aircraft within five 
nautical miles of hazardous weather conditions. Enroute to Wallops Island a good deci- 
sion was deemed to be one in which the pilot circumvented a hazardous area of convec- 
tive weather by changing course, so as to avoid the hazards by at least ten nautical miles. 
The experiment was designed to elicit the cognitive and perceptual processes involved in 
making these navigation decisions. 
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Conclusions of Experiment 


Decision Making in the Flight Environment 

When compared to the use of a cockpit weather display in a previous experiment that did 
not contain ownship position information, the introduction of ownship information did 
not improve the ratio of good decisions versus poor decisions, for either the Richmond or 
the Wallops decisions. On the other hand, the addition of ownship position information to 
the display did not have a detrimental effect on the decision-making ability of the pilot. 
Additionally, it was found that the addition of ownship position symbology reduced the 
perceived pilot workload in using the weather information display. 

The introduction of larger NEXRAD image cells had a positive, although not statistically 
significant, effect on decision making for the Richmond leg of the scenario. The introduc- 
tion of larger NEXRAD image cells did not have an effect on decision making, for the 
Wallops decision. 

Display of Ownship Position and Pilot Workload 

The introduction of ownship had a markedly positive effect on reducing the perceived 
pilot workload. This effect was attributed in large part to the reduced cognitive load re- 
quired to determine the aircraft position in relation to the hazardous weather conditions. 

Pilots’ use of Weather Information Sources 

The display of NEXRAD mosaic images substantially increased the pilots’ awareness of 
the general location of convective weather in their vicinity. The attractive visual display 
of these images, however, caused some pilots to depend too heavily on the weather dis- 
play for the information they needed regarding hazardous convective weather conditions. 
As a result, they did not feel it was necessary to obtain additional essential and corrobo- 
rating information from other available sources. 

METAR Presentation Limitations 

The METAR textual information was presented in typical ICAO teletype coded formats. 
The experiment found that the interpretation of the codes in a high workload environment 
is prone to errors. Many errors were observed and excessive fixation times were observed 
when the pilots attempted to decode the METAR information. Many of the pilots com- 
mented that the METARs would be more useful if they were displayed with their English 
translation, much as DU ATS provides the English translation. The METARs were up to 
an hour old in many cases, and although the information could be current, the perception 
of latent information may have caused the pilots to disregard the information in favor of 
ATC or pilot reports. 

Increased Situational Awareness 

The relationship between pilot workload and situational awareness was the subject of 
many of the comments received from the pilots — that the weather display slightly in- 
creased their workload, but vastly improved their situational awareness. A frequent 
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comment from the pilots’ epitomizes the issue, “The weather display took some time get- 
ting used to... but was well worth the added workload.” 

Judging Proximity and Movement of Hazardous Weather 

The age of the NEXRAD images on the weather display led to noticeable errors commit- 
ted by many of the pilots in the course of determining the proximity and rate of move- 
ment of the hazardous convective weather. 

Stimulus Area Effect 

Analysis of the Richmond decision indicated that better decisions were made with the 
introduction of larger NEXRAD image cells. One explanation for this finding converges 
on the stimulus area effect. The stimulus area effect states that the larger the visual area 
of a warning stimulus, the greater importance it holds. 

The larger stimulus area — presented by the larger NEXRAD image cell size — created a 
greater uncertainty in the exact location of the hazardous weather, which led the pilots to 
select a track farther away from the depicted weather. 


Recommendations 

The following recommendations are provided for the consideration by cockpit weather 
display system manufacturers: 

• Consider providing ownship information 

• Provide direction and rate of the movement of hazardous weather 

• Provide distance determination 

• Provide intuitive NEXRAD image age information 

• Provide METAR code english translation 


Recommended information to be added to the AIM and to Advisory Circulars include: 

• That pilots become fully proficient in determining and maintaining awareness of the 
age of data-linked weather products 

• Pilots become aware of the limitations that the age imposes 


Recommendations are also provided for further research and development efforts, 
including: 

• Improved hazardous convective weather forecast products 

• Storm cell movement depiction 

• Icing forecast products 

• Aggressive development of means for providing training in use of cockpit weather 
displays 
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1 Introduction 

Statistics indicate that there is, on average, one fatal general aviation accident per day in 
the United States alone (AOPA, 1999 Nall Report). Some of the reasons for these fatali- 
ties include pilot-related causes, mechanical failure, midair collisions and other problems. 
While mechanical failure accounts for only 14.1 percent of the total accidents, pilot- 
related causes account for over 73 percent of the total accidents. The primary causes of 
fatalities were weather, maneuvering flight and approaches. Weather-related accidents 
were more likely to be fatal than any of the other major causes of fatal accidents. With an 
overall fatality rate of 83. 1 percent, weather related accidents were the deadliest of the 
pilot-caused fatalities. Most fatalities involving weather were the result of controlled 
flight into terrain or other objects, spatial disorientation leading to uncontrolled flight, or 
pilot-induced structural failure of the aircraft. Some accidents attributed to other causes 
involved weather as a contributing factor as in the cases of improper IFR approach acci- 
dents. Windshear and crosswind also caused weather-related accidents. Most troubling is 
that 72.2 percent of the weather-related fatalities were caused by attempted VFR flight 
into Instrument Meteorological Conditions (IMC). 

While pilot training and certification regulations to minimize pilot error have been im- 
plemented, there have been significant advances in technology that can offer advanced 
weather displays in the cockpit via data link. This could provide a significant advance in 
aviation safety. Conventional round dial instruments accompanied by aeronautical charts, 
approach charts, and flight service station briefs represent a few of the many separate 
pieces of data that must be accessed for safe flight. The pilot is obliged to integrate these 
various pieces of information into a single mental model of the outside world. This repre- 
sents a very appreciable cognitive workload, and, inevitably, mistakes are sometimes 
made. 

Advances in display system design are attempting to reduce a pilot's cognitive workload 
by doing much of the integration behind the scenes. These designs are moving toward 
flat-panel displays with terrain, traffic, routing, and weather all overlaid on a single 
screen, thereby fostering a more intuitive mental model of “the big picture” for the pilot. 
By reducing the workload involved in mentally integrating multiple elements, a pilot can 
allocate attention elsewhere, particularly to higher level situation assessment, judgment 
and decision making tasks. Extra attention to these tasks should reduce the potential for 
error and enable the individual to make better decisions. 

However, because human performance research has lagged well behind the display 
manufacturers, many of the performance issues are yet to be determined, and the best 
way to display weather information is not yet clear. Nevertheless, weather information 
(because of its great importance in flight safety) is a prime candidate for early implemen- 
tation in the cockpit. 

In 1999, the Federal Aviation Administration (FAA) entered into partnerships with indus- 
try for the development of two Flight Information Services Data Link (FISDL) systems. 
The FISDL systems will broadcast text and graphical weather information products via 
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data link for reception and display in equipped aircraft. An overview of the FISDL sys- 
tems is provided in Appendix A, Flight Information Services Description. 

The initial FAA guidance for pilots is limited to a FISDL description included in the 
AIM. The guidance outlined in the AIM is provided in Appendix B. Guidance matenal 
for FISDL avionics manufacturers and service providers is available in the RTCA docu- 
ment DO-267, Minimum Aviation System Performance Standards (MASPS) for Flight 
Information Services-Broadcast (FIS-B) Data Link, dated March 27, 2001. 

The RTI International's Center for Aerospace Technology, sponsored by the FAA and 
NASA, investigated pilot performance using a prototype airborne weather display in a 
full mission simulator developed expressly for the study of new cockpit technologies in 
general aviation (Yuchnovicz, Novacek, Burgess, Heck, Stokes, 2001). The initial ex- 
periment was conducted with current instrument rated pilots who were presented with a 
challenging but realistic flight scenario involving weather with significant embedded 
convective activity. The objective of the experiment was to investigate the potential for 
misuse of weather information, and thus provide information to the FAA for use in de- 
velopment of guidance to pilots and manufacturers. 

1.1 Potential Issues with Datalinked Weather Displays 

One potentially significant issue in the use of displayed weather is that weather products 
are not displayed in real time as are most other cockpit data including the data provided 
by on-board weather radar. In the best of circumstances, the latest graphical NEXRAD 
products will be broadcast to aircraft within one minute of reception from the weather 
service provider, but will already be five or six minutes old when received from the 
weather service provider for transmission to the aircraft. 

This presents the pilot with complex issues of interpretation and prediction. It is not clear, 
for example, whether pilots will try to extrapolate, from delayed data, the current position 
of storm cells, and attempt to weave between areas of perceived danger (tactical use), or 
adopt a more conservative approach of longer-term route planning to avoid potential haz- 
ards altogether (strategic use). The term “perceived” is crucial here, as studies to date 
suggest that a “keep out of the red” heuristic procedure may be adopted when, for exam- 
ple, viewing a NEXRAD baseline reflectivity product indicating amounts of rainfall ac- 
cording to a color coding scheme. Of course, the cessation of red cells (indicating areas 
of heavy rainfall) does not imply the cessation of peril. Areas of low visibility, turbulence 
and windshear may not appear as coded zones in certain weather products so any such 
heuristic procedure is a dangerous one. 

It has been anticipated by some that pilots might try to use the data-linked weather infor- 
mation “tactically” as though it were “real time” and definitive, instead of delayed and 
probabilistic, possibly getting themselves into trouble. This might lead to pilots that be- 
come overconfident in their ability to judge exactly where it is safe or unsafe to fly. It is 
believed that “strategic” use of the weather display (using the information to plan a route 
around possible danger zones) would be safer and more appropriate. 
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A related issue concerns the explicit provision of predicted weather (e.g., storm cell con- 
figuration, location and movement), such that the mental workload of extrapolation is not 
added to pilots’ tasks. Manufacturers and regulatory agencies may be hesitant in provid- 
ing mathematical predictions and extrapolations of weather data to pilots because there 
may be non-trivial liability issues involved. 

1.2 Survey of Relevant Literature 

Pertaining to the display of data-linked weather information, relatively little documented 
research has been conducted to date. The next generation of research must begin in order 
to catch up to rapidly emerging technology. 

Past studies have primarily focused on situational awareness (Hansman, & Wanke, 1989; 
and Lee, 1990), and expert/novice strategic decision making, (mostly making go/no go 
decisions), (Driskill, Weissmuller, Quebe, Hand, Dittmar, Metrica, & Hunter, 1997; Der- 
showitz, Lind, Chandra, & Bussolari, 1996; Fisher, Brown, Wunschel, & Stickle, 1989; 
Wiggins, Connan, & Morris, 1995; and Wiggins & O'Hare, 1995). Little has been done to 
examine the possible “tactical” decisions made during flight, and none have looked at this 
issue in a full mission simulator. 

One of these issues is the impact of textual versus graphical presentation of weather in- 
formation on pilot decision making. A particularly relevant study was a comparison of 
textual presentation versus graphical presentation of weather information undertaken at 
the Lincoln Laboratory of MIT (Lind, et. ah, 1994) that provided a valuable first step by 
looking at the influence of data-link provided graphical weather on pilot decision making. 
When compared to strictly text information, the graphical information caused pilots to 
become more confident in their assessment of the weather, and to make better Go/No Go 
decisions as well as flight path change decisions. Although very valuable, this study was 
performed in an office setting without a true flight simulator and, therefore, without fac- 
tors that come into play in an operational setting. Decisions were made based on static 
images presented at selected certain points during a scripted scenario. 

Spatial displays have also been found to improve accuracy over text in presenting infor- 
mation for an analog operation/tactical decision task (Wickens & Scott, 1983). 

All these findings are consistent with the multiple resource theory of attention, and the 
proximity/compatibility principle. These findings suggest that if an individual is to per- 
form a visual-spatial task (such as navigating an aircraft through the airspace), then the 
information needed to perform that task should be presented in a visual-spatial way (e.g. 
as graphics, rather than, for example, a visual-verbal way such as in teletyped weather 
products). 

The studies performed to date represent a fraction of the studies that are needed with the 
introduction of new technologies to resolve issues that arise in implementation and opera- 
tional use. 
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2 Participants 

This experiment was a cooperative effort between the Federal Aviation Administration 
(FAA), National Aeronautics and Space Administration (NASA) and RTI International. 

2.1 FAA Data Link Office 

The FAA Flight Information Data Link Office (AUA-420) is a prime sponsor for this ex- 
periment. This effort was undertaken to support the development of guidance for the use 
of cockpit weather displays in the National Airspace System. 


2.2 NASA AWIN Project 

The NASA AWIN (Aviation Weather INformation) project is also a prime sponsor, and 
provided technical support and contract management for this experiment in partnership 
with the FAA. AWIN is an element of the Weather Accident Prevention Project of 
NASA’s Aviation Safety Program. 

2.3 RTI International 

The experiment was performed by the Flight Systems Engineering Office of RTI located 
in the Hampton, Virginia, office. An RTI consultant from Rensselaer Polytechnic Insti- 
tute assisted in the experiment design and analysis. Another RTI consultant provided air 
traffic control expertise in the design and execution of the experiment. 
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3 Methodology 

The objective of this follow-on experiment was to investigate the effect of ownship posi- 
tion information and NEXRAD cell size resolution on a pilot’s decision making ability. 

The hypothesis states that the introduction of ownship position informa- 
tion on the weather display will improve navigation decisions. Addition- 
ally, finer NEXRAD image resolution will impact navigation decisions. 

3.1 Experiment Design 

It is anticipated that with sufficient training, careful use of the cockpit weather display, 
and prudent pilot procedures in instrument flight conditions, the emerging cockpit 
weather display products will provide substantial improvements to the safety of flight. It 
is important to note that this experiment, like the one that preceded it, was designed spe- 
cifically to identify potential hazards in the use of cockpit weather displays. 

Every aspect of the design of this experiment was undertaken with this objective in mind, 
including subject pilot selection, subject pilot training, and the mission scenario. Pilots 
were selected so as to provide as wide and representative a range as possible of the ex- 
perience, knowledge of weather and risk aversion of the population of general aviation 
pilots who might use these emerging cockpit weather display products. The training pro- 
vided the subject pilots was tailored so as to provide them with sufficient familiarity with 
the experimental equipment to successfully accomplish the mission scenario, while at the 
same time creating a reasonable probability that within the population of subject pilots 
selected, potential hazards in the use of the equipment might become apparent. Likewise, 
the mission scenario incorporated in the experiment was selected to ensure that it could 
be accomplished by the average pilot with careful attention by the subject pilot to the in- 
strument flight procedures, but offered sufficient opportunity for observation of human 
error in the use of the prototype cockpit weather display where such hazards might exist. 

The experiment was designed to have certain desirable properties. It was moderate in 
length (approximately one hour depending on pilot actions) in order to eliminate fatigue- 
related effects. It was made up of sufficiently independent phases to test responses to dis- 
crete weather conditions. The incident density was to be plausible and would be designed 
to occur while crossing informational boundaries (where most decision related errors are 
more likely to occur). The mission scenario and cockpit simulator were to be sufficiently 
realistic such that the subject pilot would be immersed in the experiment. 

The experiment employed a between-subjects design, whereby two groups of similar sub- 
ject pilots were divided into control and treatment groups. Performance differences be- 
tween the two groups could then be attributed to differences between the control and 
treatment conditions. A previous Aviation Weather Information (AWIN) experiment, 
compared a control group of 12 pilots (group A) without a weather display against a 
treatment group of 13 pilots (group B) that did have a weather display. The follow-on ex- 
periment, reported herein, compared a group of 12 pilots (group C) that had ownship 
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symbology, against the 13 pilots (group B - treatment) of the previous experiment that 
did not have ownship symbology. Therefore, the control group and treatment group were 
similar except for the introduction of ownship symbology. 


Within the follow-on experiment, the performance of two groups of 12 pilots each were 
compared. The experimental treatment consisted of the introduction of reduced-resolution 
NEXRAD graphic images. The first group of 12 pilots (group C) were presented with a 
NEXRAD image cell size of 4km (same resolution used in the previous experiment), 
while the second group of 12 pilots (group D) were presented with a NEXRAD image 
cell size of 8km. The relationship of the experimental groups can be seen in Figure 3-1. 



Previous AWIN Experiment — 
Neither Group has Ownship Icon 


Figure 3-1. Relationship between previous and current experiment groups. 

In addition to determining the impact of ownship symbology and NEXRAD display reso- 
lution on pilot decision making, several additional concerns were identified, and ad- 
dressed in the course of the experiment to provide advisory information for the FAA and 
feedback to the avionics manufacturers. For example, it was noted how much instruction 
and practice was required to familiarize and operate the display. The pilots were ques- 
tioned about their understanding of the data content, refresh rate, staleness, and 
NEXRAD image cell size resolution. Post flight questionnaires were administered to de- 
termine if the pilots were able to determine both their location and their proximity to dis- 
played weather. They were also invited to comment on the functionality of the display 
and any improvements they could suggest. 
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A simulated flight was designed (using actual recorded weather) that originated in New- 
port News, Virginia, and consisted of two decision points: 1) approaching the Richmond, 
Virginia, airport, and 2) enroute to Wallops Island, Virginia. The actual weather consisted 
of two converging frontal boundaries. One frontal system included convective activity 
moving rapidly west to east across the vicinity of the Richmond airport. Another some- 
what stationary trough of low pressure included convective activity developing along a 
north/south line over the Chesapeake Bay. The first decision emphasized the time-delay 
(temporal) aspects of the weather while the second decision emphasized the spatial as- 
pects. Details of each scenario providing the two decision conditions are given in greater 
detail in the Results section of this report. 

3.2 Pilot Selection Process 

Due to the nature of the flight scenario, it was decided to limit subject pilot candidates to 
those who were instrument rated and legally current. The previous weather display ex- 
periment used risk aversion and weather knowledge as selection criteria after determining 
instrument currency. Risk aversion was measured using a PC-based test described in Ap- 
pendix C, Risk Assessment Test. The risk aversion test provides a domain independent 
measure of what a subject pilot does in response to a risk-reward opportunity, not of what 
they say they will do. 

Weather knowledge was measured with a written test, provided in Appendix D, Weather 
Knowledge Questionnaire and Key. The test was promoted as a general aviation ques- 
tionnaire to disguise the true nature of the experiment, thereby reducing any tendency for 
a subject pilot to study weather interpretation before the actual simulator trials began. 

The tests were administered to 57 current IFR-rated pilot candidates. By combining the 
risk aversion and weather test scores, the pilots were selected and organized into two 
groups that demonstrated either high-risk/low- weather-knowledge tendencies or low- 
risk/high-weather-knowledge tendencies. The approach for selecting and organizing the 
subject pilots was used to maximize the likelihood that navigation decision errors would 
be observed within the relatively small sample size of qualified pilots. 

The previous weather display experiment used 25 pilots from the initial screening. The 
results of that previous experiment showed a slightly positive correlation between 
risk/weather-knowledge and eventual decisions, although the results were not statistically 
significant. As a result, pilots were not selected for the follow-on experiment based on 
their risk/weather-knowledge scores, but were instead selected at random from the pilots 
remaining in the initial screening pool who had not been selected for the first experiment. 
Due to the pilot selection method of the previous experiment — using the extremes of 
risk/weather knowledge scores — the remaining subjects were roughly equivalent with 
respect to their risk predilection/weather knowledge scores. 

Twenty-four pilots were required to complete the follow-on experiment, but due to 
scheduling problems, seven pilots needed to be recruited beyond the initial 57 tested in 
the initial screening. These added pilots were given the same risk aversion and weather 
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knowledge tests as the initial pool of pilots to permit comparative statistical analysis. Al- 
though not intentional, all of the pilots for the follow-on experiment were male. 

3.3 Experimental Apparatus 

The experiment was performed in a full-mission flight simulator to provide a realistic op- 
erational environment. Two major components comprise the experimental system: pre- 
flight planning tools and the flight simulation facility. 

3.3.1 Pre-Flight Planning Tools 

Each pilot was given 30 minutes to plan the flight. The following flight planning tools 
were provided: 

• A written transcript of a telephone Flight Service Station (FSS) weather brief- 
ing (provided in Appendix E, Preflight Weather Briefing) 

• Aircraft Flight Manual 

• Aeronautical charts (sectional and IFR low-altitude enroute) 

• Blank flight logs 

• Partially completed flight plan forms (each pilot given same route). 

3.3.2 Flight Simulation Facility 

The flight simulation facility consisted of a full-mission simulator that provided a simula- 
tion of a complex, high-performance single-engine, single-pilot IFR-equipped airplane 
having the major features and performance of a Piper Malibu PA-46-3 1 OP. The key ele- 
ments of the simulation facility are illustrated in Figures 3-2 and 3-3. This full-mission 
simulator facility consisted of three major sections as follows: 

• Aircraft Cockpit Simulator - Consisted of the cockpit mockup with controls, instru- 
ments, radios and indicators. A closed-circuit television camera was mounted behind 
and above the pilot’s left shoulder to provide live images from the cockpit to the Sce- 
nario Controller and Observer positions. The simulated cockpit instrumentation is 
shown in Figure 3-4. The weather information display was located between the pri- 
mary and secondary instruments to maximize its visibility and probability of use. 

• Simulation Facility and Scenario Controller and Observer Positions - Consisted of 
the master control station used for scenario generation and for selection, monitoring 
and recording of flight progress. It provided the operator with displays of all control 
positions, radio and instrument switch positions, instrument displays and the Out-the- 
Window scene (as presented to the subject pilot). A weather data display consisting of 
NEXRAD images was provided for the scenario controller, and enabled the observer 
to track the flight’s progress relative to the weather. A video image of the cockpit 
from the camera was provided to enable the observer to monitor the subject pilot’s ac- 
tions. Five audio of all radio transmissions between the pilot and the Air Traffic Con- 
troller, Flight Watch, ATIS, etc., was available to the simulation scenario controller 
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and to the observer. An intercom audio network was provided which permitted pri- 
vate conversations between the scenario controller, observers, and air traffic control- 
ler positions. The ability for the pilot and air traffic controller to communicate was 
also provided by the same intercom system. All intercom traffic was recorded on the 
audio track of the video recording. 

• ATC Controller Position - Consisted of a custom ATC workstation developed for ex- 
periments of this type and a weather display that provided the latest NEXRAD im- 
ages enabling the ATC controller to track the flight’s progress relative to the weather. 
The NEXRAD images were updated every minute. A display of the current pilot- 
selected communication frequencies was also provided so that the ATC controller 
could verify that the pilot was contacting ATC on the correct frequency before re- 
sponding to an initial contact. 

Additional detail regarding the experimental system is provided in Appendix F, Cockpit 

Research Facility Description. 



Figure 3-2. Experimental System Facility Layout 
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Figure 3-3. Key Simulation Facility Stations 
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3.3.3 Cockpit Weather Information Display 

The weather information display system used in this experiment consisted of two key 
components, a PC based computer and a display unit, both obtained from Honeywell 
(previously NavRadio Inc.). 

The computer was a PC based workstation running Microsoft Windows NT and custom 
software. The PC was used to record, process and playback the NEXRAD/METAR data 
gained through a C-band satellite downlink receiver. Using SkyForce software, the PC 
sequenced through the database of previously recorded NEXRAD/METAR information 
and displayed the images on the SkyForce Observer. The PC also contained the software 
to depict the moving map, mode/scroll control, airport/navaid information and map data- 
base. 

The SkyForce Observer display unit contains a flat-panel LCD display of 320 x 240 reso- 
lution, a joystick and mode select softkeys. The Observer provides only the display and 
control function of the system. The computational effort resides in the PC workstation. 
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The SkyForce Observer display enables the pilot to select the weather information and 
contains the following features: 

• NEXRAD image depiction 

• METAR text reports 

• Graphic symbology of airports, navaids, major highways and state boundaries. 

• Depiction of graphic METAR symbology 

• Zoom and scroll capability 

• Map range scale 

• NEXRAD image timestamp 

The placement and layout of the SkyForce Observer controls are depicted in figures 3-5 
and 3-6. Details of the FISDL system, of which this display is an early prototype, are 
provided in Appendix A, Flight Information Services Description. 



Joystick 




Scroll 


NEX/MTR 


IlMenu OH 


Brig htness 
Control 


Figure 3-5. Weather Display Controls 
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Figure 3-6. Weather Display Screen Labels 


The NEXRAD image employs a six-color palette to depict the precipitation returns for a 
given area. The pilot was able to zoom and scroll around the image to the desired view. 
The arrangement of the intensity levels is shown in Figure 3-7. The six-color palette 
represents the NEXRAD precipitation intensities received and used in the experiment. 
Subsequent to the initiation of this experiment, RTCA guidance (DO-267) was published 
for FIS-B systems, further limiting the color palette used to depict NEXRAD images. 


NEXRAD Intensity Levels: 

Lt. Green ill 

Light 

Dk. Green WM 
Yellow | 

Dk. Yellow 111 
Red HH 
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Severe 


Figure 3-7. NEXRAD Mosaic Image Precipitation Intensity Key 
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The 10 small-cell NEXRAD images used in the experiment are duplicated in Appendix 
G, and the large-cell NEXRAD images used in the experiment are duplicated in Appen- 
dix H. 

The graphical METARs are small graphic squares that depict the ceiling and visibility for 
the reporting station. The coding of the graphic METARs are shown in Figure 3-8. 


CEILING 

Red - Less than 500 ft 
Yellow - 500 to 1000 ft. 
Green - 1 000 to 3000 ft. 
Blue - More than 3000 ft 


^ 

VISIBILITY 

Red - Less than 1 mi. 
Yellow - 1 to 3 mi 
Green - 3 to 5 mi. 

Blue - More than 5 mi. 



Red - Low IFR 
Yellow- 1FR 
Green - Marginal VFRI 
Blue -Unlimited 


Figure 3-8. Graphic METAR Key 
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4 Procedure 


4.1 Key Phases of Experiment Procedure 

The experiment procedure consisted of the following five phases: 

1 . Experiment briefing 

2. Simulator familiarization 

3. Pre-flight planning 

4. Simulator mission 

5. Post-mission briefing 


4.1.1 Experiment Briefing 

The subject pilots were given a 45-minute briefing of the mission objective, mission sce- 
nario, and an overview of the simulator controls. The pilots were also given an overview 
of the weather display operation. The briefing scripts are provided in Appendix I, Ex- 
periment Briefing. 

4.1.2 Simulator Familiarization 

The subject pilots were provided with a familiarization session and practice flight in the 
simulator. Systems, controls and displays were explained and demonstrated. The simula- 
tor instructor answered any questions that the pilot had with respect to the operation of 
the simulator. Additionally, the pilots were given a hands-on training session on the use 
of the in-flight weather display system and the autopilot. 

The training was provided in an interactive environment that gave a thorough understand- 
ing of the equipment and its capabilities. To assure equal treatment to all subject pilots, 
the training session was heavily scripted and the pilots were trained to a predetermined 
performance level derived from the FAA Practical Test Standards for Instrument Pilots. 

The simulator training took approximately 45 minutes and included a practice flight with 
two approaches. The training script and proficiency criteria can be found in Appendix J, 
Simulator Briefing and Training. 

4.1.3 Pre-Flight Planning 

Each pilot was given 30 minutes to plan the mission. Weather reports and flight planning 
materials were provided. Additionally, a partially completed flight plan form was pro- 
vided that had the route and aircraft-specific particulars completed. The pilots were told 
that the route given to them was already filed with Flight Service, but that they were free 
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to change the route upon the initial call to clearance delivery. The flight plan route is il- 
lustrated in Figure 4-1. They were also told that all the normal weather information ser- 
vices typically available in the National Airspace System (NAS) are also available in the 
simulator, including Flight Service, Flight Watch, ATIS, ASOS and ATC. 



Figure 4-1. Flight plan route given to pilot. 

4.1.4 Simulator Mission 

The pilots were left alone in the simulator for the mission and observed remotely. The 
mission lasted approximately one hour, depending on the pilot and route selected around 
the hazardous weather conditions. 

A team of four individuals, including a Simulator Operator, an Air Traffic Controller, an 
Observer/Director and a second observer conducted the experiment. The Observer’s pri- 
mary role was to collect and record data on the Observer form (Appendix K), including 
the comments of the other members of the research team. Additionally, the Air Traffic 
Controller provided a record of his observations that are included in each subject pilot’s 
data folder. 
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4.1.5 Post-mission De-briefing 

Upon completion of the mission, each pilot was given a questionnaire while still seated in 
the simulator, thus providing important subjective comments while still in the mission 
“mode.” The questionnaire is included in Appendix L, Immediate Reactions Question- 
naire. 

After completing the questionnaire, the pilots were interviewed by the experiment ob- 
servers using the Structured Interview Guide (Appendix M) to confirm behavioral actions 
and decisions. 

As a final step, the pilot completed an open-ended questionnaire. The questionnaire is 
included in Appendix N, Weather Display Questionnaire. 

4.2 Flight Procedures 

The mission flown by the subject pilots consisted of two flight legs, the Richmond leg 
and Wallops Island leg. 

4.2.1 Mission Scenario 

The mission scenario consisted of a flight to deliver medication to a diabetic patient at the 
NASA Wallops Island facility. The NASA Wallops Island facility is located on the east- 
ern shore of Virginia. The pilots were told that the insulin available in the Wallops area 
had become tainted and a new supply was to be flown to the patient. They were also told 
that diabetic ketoacidosis (DKA) is a common and potentially fatal complication for 
which one effective therapy is a special form of sodium bicarbonate. Thus, the medical 
rationale involved the delivery of a vital medication, insulin, and a desirable medication, 
sodium bicarbonate. The pilot was informed that a medical mercy mission had been co- 
ordinated and that he was to be the pilot. 

The flight originated at the Newport News Virginia airport, with the insulin already on- 
board the aircraft. The pilot was instructed to fly to Richmond Virginia and pick-up the 
special sodium bicarbonate on the way to Wallops Island. 

In the course of the preflight briefing, the pilot found that there was a weather front mov- 
ing into Richmond, but that the forecast for the area would permit the pilot to land at the 
Richmond airport to pick-up the sodium bicarbonate medicine. The forecast weather for 
the entire flight placed the aircraft in instrument meteorological conditions, but the 
weather at Wallops Island airport was forecast to be above minimums. 

All flights were flown in a full-mission simulation facility in simulated instrument mete- 
orological conditions. Visibility for the pilot was essentially zero from shortly after take- 
off until just before landing. The pilots were to conduct the flight in accordance with all 
appropriate ATC procedures in conjunction with an Air Traffic Controller (ATC), located 
in an adjoining room. The ATC workstation fulfilled the roles of clearance controller, 
ground controller, tower controller, approach/departure controller and FSS briefer as re- 
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quired throughout all phases of the flight. The scripts are provided in Appendix O, Air 
Traffic Control Scripts. 

The pilot was able to access the normal in-flight weather services through the simulator 
radios, including: 

• FSS - Flight Service Station 

• ATC - Air Traffic Control (tower, departure and approach) 

• FW - Flight Watch 

• ATIS - Automatic Terminal Information Service 

• ASOS - Automated Surface Observation System 

The ATC workstation presented the Air Traffic Controller with a readout of the fre- 
quency that the subject pilot selected on the simulator communication radio. When the 
pilot tuned the communication radio to a frequency that corresponded to a recorded 
weather message (ATIS, etc.), a prerecorded report was played through the intercom. The 
ATIS/ASOS recorded scripts can be found in Appendix P, Enroute Weather Report 
Scripts. 

If the pilot called either a Flight Service Station or Flight Watch briefer, the Air Traffic 
Controller read a scripted weather report to the pilot depending on the time of the call. 
These weather scripts can be found in Appendix P, Enroute Weather Report Scripts. 

Actual weather data was used to assure the realism of the operational scenario. All 
weather information used in this experiment was recorded from actual weather conditions 
that existed in the geographical area of the experiment on the evening of April 25, 2000. 
The NEXRAD images were recorded during passage of multiple weather fronts through 
southeastern Virginia from a prototype satellite data gathering system provided to RTI 
International by Honeywell, Inc. The NEXRAD images were replayed on the weather 
display in the simulation facility cockpit. 

All NEXRAD mosaic images used in the experiment were recorded with a cell resolution 
of 4 km. Half of the pilots (12) received the 4 km NEXRAD image cell sizes, while the 
second group of 12 received 8 km cell sizes. Special software was used to systematically 
reduce the resolution of the images in a consistent manner. The small-cell (4x4 km) 
NEXRAD images were processed through a software filter that assigned the highest level 
radar return to the entire large cell. 

To realistically reproduce actual data-linked weather products, the subject pilot received 
the NEXRAD mosaic images delayed by seven minutes. The pilot’s weather display of 
NEXRAD images were initially seven minutes old, aging to 14 minutes old before receipt 
of the next update (of a seven minute-old image). The pilot also had access to graphical 
and textual Aviation Routine Weather Report (METAR) information. 
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The NEXRAD weather display used by the Air Traffic Controller emulated the ASR-9 
weather radar. The Air Traffic Controller received a real-time NEXRAD image that was 
no more that one minute old. 

All the other weather data products needed to develop preflight and inflight weather re- 
ports for the experiment scenario were collected from the appropriate FAA sources for 
the same location, date and time captured in the NEXRAD mosaic images. 

4.2.2 First Leg of Flight - Newport News to Richmond 

During the course of the first leg of the flight, between Newport News and Richmond, the 
ceiling and visibility at the Richmond airport had descended to below minimums (200 
feet) sooner than forecasted. Additionally, there was a thunderstorm approaching the 
Richmond airport. The only way the pilot could leam of these deteriorating conditions 
was to obtain an in-flight update of the weather. The pilots could gather these updates 
either through the weather display or by radio. 

Before reaching the initial approach fix for the Richmond airport, the weather display de- 
picted a thunderstorm cell several miles to the west of the airport but headed toward the 
airport (see Figure 4-2). The image on the pilot’s weather display was a minimum of 
seven minutes old and would have aged up to 14 minutes. By the time the pilot began the 
approach, the actual weather cell had intensified and moved closer to the airport. [The 
ATC workstation weather display showed the storm to be approximately two miles 
northwest of the airport.] 



Figure 4-2. Display Image When Approaching Richmond 
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There were several possible responses to this scenario. The pilot could continue the ap- 
proach with old data and proceed into the thunderstorm (poor decision), or, the pilot 
could decide to abandon the approach into Richmond and proceed directly to Wallops 
Island (good decision). A third option was for the pilot to ask ATC to provide a hold until 
updated weather information could be obtained and sorted-out before deciding to con- 
tinue into the Richmond airport or proceed to Wallops Island (good or poor decision de- 
pending on proximity of flight path to thunderstorm). To preserve the timing aspects of 
the experiment, the pilots were told that the medicine needed to be delivered to Wallops 
Island within one hour. 

As the aircraft traversed the various precipitation zones — as depicted on the simulator 
operator’s NEXRAD image display — the simulator operator introduced levels of turbu- 
lence appropriate to the precipitation level. For flight in clear air, turbulence was not en- 
countered, but when the aircraft traversed into an area depicting precipitation, a turbu- 
lence model was applied to the simulation and the turbulence was increased in proportion 
to the intensity level. 

If the pilot gathered weather information (either via voice or from the weather display) 
during the leg between Newport News and Richmond, the pilot was apprised of the rap- 
idly changing weather and had to make a decision to either divert to Wallops Island or 
continue the approach into Richmond. This is the decision that the experiment was de- 
signed to uncover along with the basis for the subject pilots’ decisions. 

If the pilot proceeded with the approach into Richmond, typical and consistent weather 
warnings were given to the pilot by ATC, including a windshear warning when the pilot 
contacted the tower. To expedite the simulator mission, if the pilot decided to proceed 
with a landing into Richmond, ATC informed the pilot (when crossing the final approach 
fix) that the Richmond airport manager had closed the airport due to windshear and heavy 
lightning activity. This methodology would preserve the timing essential to maintaining a 
consistent relationship between the aircraft position and weather movement for all the test 
subject flights. Therefore, all the pilots either broke off the attempt to land at Richmond 
at various distances from Richmond, or were waved-off at the final approach fix. 

The Newport News - Richmond leg of the experiment was designed primarily to deter- 
mine the pilot’s judgment relative to the temporal issues in the use of the weather infor- 
mation display with respect to ownship position symbology and NEXRAD image cell 
size resolution. 
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4.2.3 Second Leg of Flight - Richmond to Wallops Island 


During the leg between Richmond and Wallops Island, a line of storm cells materialized 
across the direct route to Wallops Island, with one storm cell to the north of the direct 
course and one to the south. The location of this convective activity can be seen in Figure 


4-2. 


The distance between the red cells was approximately 10-12 miles. The hole between the 
storms was tempting enough to create a corridor between the areas of hazardous weather. 
These cells did not move substantially with succeeding NEXRAD images, but slightly 
changed shape and size. 

The METAR graphical and textual depiction showed that the Wallops Island airport was 
above minimums, therefore giving the pilot an incentive to proceed with the flight to 
Wallops Island. 

The pilot was monitored as to the decision to proceed between the storm cells, or circum- 
vent the area of thunderstorm altogether. This part of the experiment was designed pri- 
marily to determine the pilot’s judgment relating to spatial issues in the use of the 
weather information display. 
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5 Results 


5.1 Subject Pilots 

A total of 24 pilots participated in this experiment. Of these 24 pilots, 12 flew with a 
NEXRAD image cell size of 4 km, and 1 2 flew with a NEXRAD image cell size of 8 km. 
Both groups had ownship position, whereas in the previous experiment the weather dis- 
play had 4 km cell size without ownship position depicted. Participants were pilots quali- 
fied for and current in instrument conditions with varying levels of flight experience, as 
illustrated in Table 5-1. 


Table 5-1. Pilot Flight Hours 


Pilot 

# 

Treatment 
Group 
(cell size) 

Total 

Flight 

Hours 

Est. 

Actual 

Inst. 

Hours 

Simulated 
Inst. Hours 
(Hood) 

Simulated 
Inst. Hours 
(Simulator) 

Inst. Hours 
Proficiency 
(Last 90 
days) 

RAT 

Score 

WX 

Test 

Score 

25 

Small 

560 

50 

56 

2 

2 

3.8 

30 

37 

Small 

2512 

204 

118 

417 

0 

3.95 

26 

15 

Small 

1880 

400 

43 

1 

9 

3.65 

21 

40 

Small 

5315 

516 

263 

121 

3 

3 

18 

11 

Small 

307 

8 

58 

0 

15 

3.55 

33 

27 

Small 

951 

76 

105 

4 

4 

4.6 

28 

9 

Small 

700 

150 

50 

0 

10 

4.2 

26 

12 

Small 

875 

30 

15 

2 

5 

3 

24 

1 

Small 

487 

13 

27 

20 

2 

4.05 

28 

56 

Small 

1159 

23 

91 

3 

2 

3.35 

27 

22 

Small 

1760 

106 

148 

3 

2 

4 

28 

4 

Small 

1525 

75 

117 

8 

9 

3.75 

30 

28 

Large 

1750 

92 

105 

10 

6 

3.55 

32 

46 

Large 

15278 

1634 

150 

350 

25 

4.35 

30 

58 

Large 

1000 

105 

210 

2 

0 

3.5 

24 

17 

Large 

20614 

2000 

20 

400 

150 

3.4 

31 

13 

Large 

536 

60 

60 

0 

6 

3.95 

32 

59 

Large 

3740 

555 

184 

350 

2 

3.45 

18 

16 

Large 

5060 

179 

182 

0 

4 

3.5 

30 

60 

Large 

1500 

38 

88 

8 

10 

4.3 

31 

62 

Large 

3000 

250 

150 

15 

30 

2.55 

31 

63 

Large 

3100 

300 

300 

0 

1 

3.6 

28 

64 

Large 

180 

0 

56 

20 

7 

3.75 

26 

65 

Large 

1100 

100 

58 

22 

10 

4.65 

30 

Average: 

3120 

290 

110 

73 

13 

3.73 

27.6 

Standard 

deviation: 

4736 

487 

74 

139 

29 

0.49 

4.05 

RAT Scores — Higher score signifies higher risk predilection. 
Weather Test Scores — 39 points possible. 
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5.2 Richmond and Wallops Island Decisions 

The results of the experiment were organized around the two key decision points estab- 
lished in the experiment procedure — the “Richmond decision” and the “Wallops Island 
decision.” 

Both the Richmond and Wallops Island decisions were scored on a 1-4 ordinal scale, 
with a (1) being a strong “poor” decision, and a (4) being a strong “good” decision. A 
score of (2) was considered a “poor” decision with good elements, while a score of (3) 
was considered a “good” decision with poor elements. The intent was to produce defini- 
tive guidelines that can be applied to each scenario using a consistent method. 

5.2.1 The Richmond Decision 

A good decision — a score of (3) or (4) — was deemed to be one in which the pilot decided 
to divert to Wallops Island prior to the Final Approach Fix (outer marker) of the approach 
into the Richmond airport, thus avoiding the hazardous weather by at least five nautical 
miles. A poor decision — a score of (1) or (2) — was deemed to be one in which the pilot 
continued with an approach past the Final Approach Fix into the Richmond airport for 
whatever reason, placing the aircraft within five nautical miles of hazardous weather 
conditions. Hazardous weather was established to be a red NEXRAD mosaic image cell, 
a known area of hazardous turbulence, or a known area of hazardous windshear. A mini- 
mum separation of 5 miles from the most hazardous part of convective weather depicted 
in a NEXRAD image (red cells) was selected as the criteria for this segment of the sce- 
nario because: 

a. The hazard is a rapidly moving and fairly localized thunderstorm with a well-defined 
leading edge. 

b. The weather conditions five miles and greater to the east of the thunderstorm was 
known to be reasonably safe with no significant turbulence. 

c. The medical scenario created a motivation to proceed to within a reasonable but safe 
distance. 


The “Richmond decision” required the subject pilot to decide whether or not to attempt to 
land at the Richmond airport in the face of a fairly rapidly moving thunderstorm passing 
within a mile or two to the north of the airport. There were a total of 1 1 different 
NEXRAD mosaic images displayed to the pilot, updating in 7-minute intervals. Figure 5- 
1 depicts the NEXRAD mosaic image at 1914Z. This image was present on the weather 
display upon arrival in the vicinity of the Final Approach Fix to runway 34 (outer 
marker) to the southeast of the Richmond airport. 
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Figure 5-1. NEXRAD image at 1914Z. 


Because of the delay in transmission of the image to the aircraft, the data was at least 7 
minutes old. Actual conditions at the Richmond airport in the time frame of this decision 
can be seen in Figure 5-2 which provides the NEXRAD image for 1 92 1Z. 



Figure 5-2. NEXRAD image at 1921Z. 
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The images depicted in these figures were the 4 th and 5 th images in the sequence of 1 1 
images used. The thunderstorm seen to the northwest of Richmond is the storm that was 
designed to elicit a weather decision from the pilots. [The actual recorded storm was 
somewhat smaller than the one depicted to the pilot and was enlarged with photo retouch- 
ing software.] This particular storm moved from west to east across the successive 
NEXRAD images at approximately 40 nautical miles per hour in the early images. The 
rate of movement of the storm diminished to less than 10 nautical miles per hour in the 
later images. 

The four-point grading criteria of the Richmond decisions are defined as follows: 

1 = The pilot continued the approach into poor weather and was waved off the ap- 

proach (from the tower controller) at the Final Approach Fix (outer marker). 

2 = The pilot abandoned the approach less than five (5) miles outside of the outer 

marker, but flew within five (5) miles of a red NEXRAD image cell, while in 

the Richmond area. 

3 = The pilot abandoned the approach by their own decision less than five (5) miles 

outside of the outer marker, and flew more than five (5) miles from a red 

NEXRAD image cell, while in the Richmond area. 

4 = The pilot abandoned the approach more than five (5) miles outside of the outer 

marker, and flew more than five (5) miles from a red NEXRAD image cell. 


5.2.2 The Wallops Island Decision 

The “Wallops Island decision” required the subject pilot to decide whether to proceed as 
first cleared to Wallops Island or detour around the hazardous weather. To proceed as 
cleared, the pilot would have flown between two thunderstorms located between Rich- 
mond and Wallops Island. Figure 5-3 provides the approximate image that the weather 
display depicted after the pilot departed the vicinity of the Richmond airport, enroute to 
Wallops Island. 
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Figure 5-3. NEXRAD image at 1935Z. 


In this image, there is a line of convective activity over the Chesapeake Bay, between 
Richmond and the Wallops Island airport. Within this line of convective activity are two 
thunderstorm cells that did not move significantly in position, but that changed shape and 
size slightly between NEXRAD images. [These two thunderstorm cells were enhanced 
slightly with photo retouching software to create an enticing corridor that tempted the 
pilots to fly between them.] 

A four-point scale was again used to grade the pilot decisions relating to the thunder- 
storms over the Chesapeake Bay. A good decision — score of (3) or (4) — was deemed to 
be one in which the pilot circumvented the hazardous area entirely by changing course to 
the south, so as to avoid it by at least ten nautical miles. The pilot would then proceed up 
the coast of Virginia to the Wallops Island airport. A poor decision — score of (1) or (2) — 
was deemed to be one in which the pilot decided to find his way around (or through) the 
thunderstorms in an attempt to proceed by the most direct route to the Wallops Island air- 
port, and for whatever reason, coming within ten nautical miles of hazardous weather. 


29 





The four-point grading criteria of the Wallops decisions are defined as follows: 


1 = The pilot flew within 10 miles of a red cell while circumventing the storms over 

the bay using the pilot’s own route planning. 

2 = The pilot flew within ten miles of a red cell, but only because of a delayed turn 

or distraction, but the intent was to circumvent by at least 10 miles. 

3 = The pilot flew within 10 miles of a red cell, but was following vectors from ATC 

and for whatever reason, ATC vectored the pilot to within that 10 miles. 

4 = The pilot avoided a red cell by 10 miles or more. 


An attempt to take a route to the north around the convective activity was also deemed to 
be a poor decision as the convective area continued into extensive restricted airspace that 
was in use and not available to the pilot. 

The four-point grading scale was designed such that if a decision in the previous weather 
display experiment (two-point scale) were to be graded by this follow-on experiment 
scale (four-point scale), the outcome would be consistent. For example, if a pilot broke 
off the approach to Richmond within three miles of the outer marker, the two-point scale 
would assign a “poor” decision and the four-point scale would assign a grade of (2), 
which is still within the guidance of a “poor” decision. 

The new scoring scheme permitted the results of this follow-on experiment (treatment 
group) to be compared to the results of the previous weather display experiment (control 
group), while still providing a graduated scale for statistical analysis in the follow-on ex- 
periment. By using this four-point grading method, more powerful analysis methods 
could be applied when comparing the decisions with other data such as weather scores, 
risk scores, and the information provided by the questionnaires. 

The results of both decision points are illustrated in Table 5-2. 
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Table 5-2. Decision Results 


Subject 

Number 

Display 

Condition 

Richmond 

Decision 

Wallops 

Decision 

25 

Small Cells 

4 

4 

37 

Small Cells 

1 

2 

15 

Small Cells 

1 

1 

40 

Small Cells 

4 

1 

11 

Small Cells 

3 

3 

27 

Small Cells 

1 

1 

9 

Small Cells 

1 

4 

12 

Small Cells 

1 

1 

1 

Small Cells 

1 

1 

56 

Small Cells 

1 

4 

22 

Small Cells 

4 

2 

4 

Small Cells 

4 

4 

28 


1 

1 

46 

KHa 

3 

2 

58 

IHEHIEISS-IIOHf 

4 

1 

17 


4 

2 

13 


1 

4 

59 


4 

1 

16 


4 

4 

60 


4 

3 

62 


4 

3 

63 


4 

4 

64 

IHSEUglEHl 

1 

1 

65 

MUEISsiiiM 

4 

1 


1 = strong poor decision 

2 = poor decision 

3 = good decision 

4 = strong good decision 


5.3 Results of Immediate Reaction Questionnaire 

Upon completion of the simulator session, each subject pilot was given a questionnaire 
(Appendix L, Immediate Reactions Questionnaire) to obtain their immediate reactions. 
The pilot was given this questionnaire while still seated in the simulator, thereby reducing 
distraction issues and obtaining valuable subjective information while the pilot was still 
in the “flight mode.” 
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Except for questions 10 and 11, there were five possible answers on a Likert scale that 
ranged from Disagree (score of 1) through No Opinion to Agree (score of 5). This ques- 
tionnaire was reviewed with the pilot during the post-flight briefing to verify that the pilot 
understood the questions and to clarify any ambiguous answers. 

The questionnaire responses are presented here in their entirety, but correlations to rele- 
vant data are included in succeeding sections. 


Question 1 . This question was asked primarily to determine the extent to which the sub- 
ject pilots “bought into” the medical scenario presented to the pilots. 

I took the medical emergency scenario seriously, in the sense that I factored 
the emergency into my decision making. 

(mean score of 4. 79 , standard deviation of 0.41) 

A score of five (5) was indicative of an “agree. ” A mean score of 4. 79 and standard de- 
viation of 0.41 with this question signifies that the vast majority of the pilots “bought 
into ” the medical scenario, thus reducing the likelihood of invalid data due to an unreal- 
istic situation. 


Question 2 . This question explored the pilots’ perception of the latency in the weather 
information, and the extent to which the weather display depicted weather in “real-time.” 

An advantage of the onboard Weather Display was showing the weather in 
real-time, that is, as it actually was at that moment. 

(mean score of 3.83, standard deviation of 1.49) 

The mean score of 3.83 indicates that the pilots were generally unaware of the latency of 
the weather information. The relatively large deviation around the mean (SD=1.49), in- 
dicated a wide variation of the perception of the degree of latency among the pilots. 


Question 3 . This question explored the pilots’ perception of the degree to which they felt 
they were able to interpret the weather display and the extent to which the display influ- 
enced their decision making. 

I attribute much of my decision making to my interpretation of the 
Weather Display. 

(mean score of 4.54, standard deviation of 0.88) 

The mean score of 4.54 (a 5 being “agree ”), combined with a small deviation (SD=0.88), 
indicates that the weather display played a large part in the decision process during the 
flight. 
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Question 4 . This question asked the pilots their perception of the extent to which they 
used all available weather information sources. 


I tried to systematically sample all sources of weather information 
open to me. 

(mean score of 4.0, standard deviation of 1.02) 

The mean score of 4. 0 indicates that the pilots thought that they did sample all the 
sources of weather information that was available. 


Question 5 . This question explored the pilots’ perceptions as to their willingness to de- 
pend solely on the weather display without cross-checking with information from other 
sources. 


I used the Weather Display but felt the need to cross-check or verify my 
conclusions from conventional weather sources (ATC, etc.) 

(mean score of 4.33, standard deviation of 0.92) 

The mean score of 4.33 indicates that the pilots used the weather display, but also 
checked it with other sources. 


Questions 6 and 7 . Two questions were asked of the subject pilots about their comfort 
with and reliance on the autopilot. 

I felt comfortable with the autopilot, in terms of understanding its 
use and operation. 

(mean score of 4.42, standard deviation of 0.88) 

Without the autopilot, my completion of the flight would have 
been compromised. 

(mean score of 4.38, standard deviation of 1.13) 

Most of the pilots said they felt comfortable with the autopilot and relied on its use to re- 
duce the workload. The autopilot was, on average, in use for 85 percent of the flight (for 
all pilots). Many of the pilots stated that without the autopilot , they would have suc- 
cumbed to an early termination of the flight. 


Question 8 . When asked about the validity of the weather display, the reactions consis- 
tently held the display validity in high regard. 


The degree of validity of the weather data appearing on-screen was a 
factor I felt I held in mind as I flew. 

(mean score of 4.58, standard deviation of 0. 72) 
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Question 9 . The pilots were asked if they regularly referred to the timestamp of the 
weather information. 

I have been monitoring the weather display time stamp very regularly in 
my instrument scan. 

(mean score of 3. 79, standard deviation of 1.47) 


Question 10 . The pilots were asked what they perceived the weather conditions to be near 
the Richmond airport. 

At the time of my arrival at Richmond’s Airport, I knew that there 
was a storm — (circle one) 

a. about 10 nm North West of the field. (5 selections) 

b. about 5 nm North West of the field. (1 selection) 

c. near the field. (10 selections) 

d. right at the field. (7 selections) 

Most of the pilots correctly perceived that the storms were very close to the airport. 


Question 1 1 . The pilots were asked what they perceived the weather conditions to be on 
the route to Wallops Island. 

At the time I was en route to Wallops, I saw across my path of direct flight, 
what I took to be — 

a. a penetrable storm. (2 selections) 

b. a navigable opening between convective cells. (6 selections) 

c. a non-navigable opening between cells. (0 selections) 

d. a wall of convective activity requiring diversion. (16 selections) 

The majority of the pilots correctly perceived the storms as a hazardous area that re- 
quired a diversion. This perception, however, did not necessarily contribute to a good 
decision. 
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Question 12 . The pilots were asked of the positional accuracy of the ownship position on 
the weather display. 


On the weather display, I found the positional accuracy of the aircraft icon 
to be adequate. 

(mean score of 4.13, standard deviation of 0.90) 

This high “agree ” score correlated with the comments received from the pilots during 
the interview — that they welcomed the ownship icon and found it accurate and easy to 
use. 


Question 13 . The pilots were asked of the positional accuracy of the ownship position in 
relation to storm information. 

In using the weather display, I felt that I generally knew the aircraft posi- 
tion relative to any storms. 

(mean score of 4.58, standard deviation of 0. 72) 

A slightly higher “agree ” score for this question (compared to #12) revealed that the pi- 
lots trusted the display elements to be accurate relative to each other, but were not as 
confident that the ownship position was an accurate representation of their track over the 
ground. 


Question 14 . The pilots were asked if they had enough sources of weather information to 
make a confident decision. 

I felt that I had adequate sources of weather information to make 
confident decisions. 

(mean score of 4. 75, standard deviation of 0.68) 

With a high “agree ” score, the pilots felt reasonably comfortable with their decisions. 


5.4 Post-flight Weather Display Questionnaire 

At the conclusion of the post-flight interview, the final item for the pilots to complete was 
a Weather Display Questionnaire (reproduced in Appendix N). The pilots were asked 
open-ended questions to obtain full responses to various issues in the use of the weather 
information display. The following section reproduces the more significant of those 
comments, and also includes researcher comments that relate to the quantitative assess- 
ments. 
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Question 13. In using this weather display today, did you find the operation 
straightforward? If not, what operations of this weather display did you find diffi- 
cult? 


“Toggling between operating modes got to be frustrating when the workload 
was high.” 

“No, locating airport of choice and obtaining METAR.” 

“I found it difficult to keep track of cells, especially movement.” 

“Switching from airport to navaids using the crosshairs — also forgetting display 
is a north-up presentation.” 

“Mostly very easy, a little trouble with crosshair info; there was different info in 
different modes.” 

“Had difficulty in pulling up text of destination airport weather in high workload 
condition while flying single pilot autopilot IFR.” 

“Would like fast way to pop-up airport/VOR identifiers.” 

“waypoint identifiers...” 

The responses to this question highlight the issue of mode selection and the difficulty that 
the pilots had in gathering information in a timely manner , especially during high work- 
load. 


Question 14. In using this weather display today, did you find the graphical 
METAR symbology useful? If not, what features did you find difficult? 

“No, the symbols were small and I had a hard time seeing them to the degree of 
discerning colors.” 

“I was not able to follow it...” 

“Graphical METAR is useful, but time lag can defeat the purpose.” 

“Glanced at it, but it was never part of my decision making.” 
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“The METAR symbols would take some getting used to during real-world flying; 
I think I would like them, but they should follow airport ID’s.” 

“Data too old to be useful.” 

The pilots did not seem to use the graphical MET ARs to a great extent , either because of 
their limited information, difficulty in interpreting the display, or inability to present 
timely information. 


Question 15. In using this weather display today, did you find the textual METAR 
presentation useful? If not, what features did you find difficult? 

“Yes, more so than colors, since it gave me a more complete picture.” 

“Useful in making decisions about selecting an alternate [airport].” 

“It is useful — might tend to delay getting ATIS.” 

“Yes, we still need to study METAR abbreviations.” 

“Yes, but would like the decoded version as an option.” 

“Yes, a plain english version would be nice.” 

“Not too useful — too much workload.” 

“...MET ARs are often too old.” 

“No, didn’t tell me anything ATIS didn’t give me.” 

The pilots found the textual MET ARs more useful than the graphical METAR depiction, 
mostly because the textual METARs contained more complete information. Additionally, 
there was still an issue of timely information and difficulty in interpreting the codes. 


Question 16. Considering your use of the weather display today, would you like to 
see any additional features or change any existing features? 

“Yes, motion vectors of weather systems (cells).” 

“Superimpose wind information for quick check and then remove.” 

“Heading-up would be more useful — GPS Nav information on the weather dis- 
play with course line would be helpful.” 
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“I would like to see the weather display to greater detail with smaller frames; 
overlay of Stormscope data desirable.” [this pilot was using the large NEXRAD 
cells] 

“Possibly a wind readout via the crosshair selection — interface with a [onboard] 
weather radar.” 

“a) I usually use my GPS at a higher magnification, 30 or 50 nm full screen. The 
highest resolution on today’s screen seemed lower. I would like higher magnifica- 
tion, even though it would be very ‘grainy.’ I think de-cluttering would help, 
b) A ‘loop’ display of radar (at least 30, 60 or 120 minutes) would help me keep 
track of what cells are doing.” 

“As currently formatted, the radar data [NEXRAD] seemed to overwhelm both 
the METAR and airport/VOR data. I would wonder about: a) Make the METAR 
and/or navigation data points larger, b) Make the radar less prominent (i.e., each 
radar ‘pixel’ not confluent with the other).” 

“I would suggest a better display of range function.” 

“It would be great to get recommended course deviation to avoid heavy weather.” 
“Some form of range information would be nice.” 

“An alert or flashing cursor of some type to indicate the NEXRAD is updating or 
has just updated.” 

“Looping for time — put storm cells into motion. Also speed and cell tops.” 

“How about ‘motion’ graphics so you can see trend better...” 

“Adding movement by repeating the last 6 or 8 frames with a ‘sequence’ button is 
worth considering.” 

“I’d like to have the moving map include the route, SUA [Special Use Airspace], 
generally include a GPS moving map.” 

“Time stamp could also include how old it is.” 

When asked if there were any features that needed improvement, most of the comments 
focused on the need for better range depiction and depiction of storm movement. Some of 
the suggestions were wind vectors, looping, motion vectors and a digital readout of cloud 
speed and tops. 
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Question 17. How did the weather display affect your decisions that you made to- 
day? 


“Very strongly influenced it.” 

“It gave me more info to base my decision making.” 

“Allowed me to immediately visualize alternate courses of action.” 

“It affected them a lot, without the display I would not have tried to hold to get 
better weather, and it helped determine routes around the weather.” 

“Was primary in decision, but I also wanted verification from another source 
(ATC).” 

“It enabled me to continue the flight.” 

“Helped me in developing an alternate course of action in determining how best 
to complete the flight assignment.” 

“Very helpful as far as situation awareness to actual weather, but not good in the 
terminal environment as a primary source of weather.” 

“Greatly influenced decisions (especially useful for rapidly changing condi- 
tions).” 

“Only for rough overview of the weather situation around me. Not useful for de- 
tails.” 

“Very helpful in overall situational awareness, but somewhat misleading in time 
critical decision making.” 

It appears that the weather information display was generally useful for strategic and 
situational awareness and poor for tactical use. 


Question 18. Do you feel that you needed more training on the weather display? If 
so, in what areas? 

“Training was enough to use it — really simple to use.” 

“Yes, perhaps not training, but just more use (it took me a few hours of use before 
I felt comfortable correlating what I see on my Stormscope with what I see when 
flying above layers).” 

“Yes, range interpretation — switching from airports to navaids using crosshairs.” 
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“Yes, pulling up text of ATIS.” 


“I would want some real life experience with it before I felt comfortable relying 
on it.” 

The pilots generally asked for more training, blit more importantly, for more time prac- 
ticing with the display in the real world. 


Question 20. Were there any features about the weather display that caused you to 
cast doubt as to its usefulness in normal, real world, operation? 

“Yes, the 7 minute delay.” 

“No, that will only come when the unit is airborne and compared to real weather.” 
“Just that the NEXRAD is at least 8 minutes old.” 

“No, provided time-lagging limitations are kept in mind. One shouldn’t expect 
anything that cannot be delivered.” 

“ATC guidance about actual conditions cast some doubt as to its usefulness.” 

The pilots commented that the time delay is a major limitation of the system. Similar 
comments were received in the previous weather display experiment. 


Question 22. How did the use of the autopilot help or hinder the use of the weather 
display? 

“...with the autopilot, I was able to more effectively use the weather display.” 
“Reduced the workload so time could be spent with the weather display.” 

“Freed up concentration so I could study the plots.” 

“Autopilot allowed me to use the weather display, higher proficiency would have 
to exist without an autopilot.” 

Because of the limited experience with the weather display and in operating the simula- 
tor, the autopilot was helpfiil and allowed time to interact with the display. This is consis- 
tent with the finding that the autopilot was in use an average of 85% of the flight. 
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6 Quantitative and Qualitative Assessments 

Quantitative assessments were undertaken to investigate any probable relationships be- 
tween the weather information display and the pilot actions at the two key decision 
points. Additionally, qualitative assessments — based on the observations and expertise of 
the experiment team — were undertaken to explore significant issues related to the 
weather information display. 

Although both decisions relate to processes within the same developing weather system, 
they are considered to be statistically independent, as considerable effort was made in the 
design of the scenario to limit coupling effects along the chronological axis of the sce- 
nario. The Richmond decision is considered to have been based largely on the interpreta- 
tion of the relationship in time between the aircraft’s expected position in the future and 
the possible location of the rapidly changing hazardous weather conditions in the fu- 
ture — a “temporal” decision. The Wallops decision is considered to have been based 
largely on interpretation of the relationship in distance between the aircraft’s flight path 
and the location of the relatively stationary hazardous weather conditions — a “spatial” 
decision. 

6.1 Subject Group Comparisons 

For the purpose of examining the effects of displaying ownship position and variation of 
NEXRAD display resolution upon Aeronautical Decision Making (ADM), an important 
consideration is the degree to which the subject pilot groups were equivalent in implicit 
knowledge (proficiency) and explicit knowledge (weather knowledge questionnaire). 

Examination of experience and proficiency in terms of the customary indices, such as to- 
tal hours, actual instrument hours, hours “under the hood,” and flight simulation hours, 
indicated there were no significant differences between the subject pilot groups. Addi- 
tionally, there were no significant differences in terms of weather knowledge or risk aver- 
sion, as indexed by the Risk Assessment Task and Weather Knowledge Questionnaire; 
these being the key variables by which the groups were matched for functional equiva- 
lence. Therefore, the groups appear to have been statistically matched in terms of the pro- 
ficiency indices used in this experiment — flight hours, instrument flight experience, cur- 
rency, weather knowledge and risk aversion. 

An indicator of experiment acceptability is the extent to which the subject pilots internal- 
ized the mission scenario. A strong acceptability from the pilots verified that the scenario 
was realistic and resultant data could be relied upon. 

6.2 Quantitative Assessments 

The form of some of the data points collected in the experiment, such as binary decisions, 
favored non-parametric tests, such as the Chi-Square test. This technique was used in 
testing the relationship of several variables to decision making adequacy, and provided a 
value for chi together with the probability (p) that a result was due purely to chance. A 
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less than a one in twenty chance that a result was merely a random fluke (p< 0.05) is gen- 
erally regarded as statistically significant. This report adheres to that convention. It does 
not, however, imply that small probability outcomes in excess of 0.05 are without impor- 
tance or necessarily result from random chance. 

Other data points were continuous numeric data, such as weather test scores, flight hours, 
questionnaire responses (a 1 to 5 Likert scale), etc. An SPSS Inc. software program was 
used to analyze the data set. Pearson Product Moment correlation coefficients were calcu- 
lated to evaluate the relationships among the continuous numeric data, and a regression 
analysis was conducted on the data for the two decision points. 

As described earlier in the Methodology section, the experiment scenario featured two 
critical decision points, one on the approach to the Richmond airport, and one en-route to 
the Wallops Island airport. These decisions were considered both independently and to- 
gether as a group. Additionally, the decisions of the current experiment were compared to 
the previous experiment, especially with regard to the effect of including ownship posi- 
tion symbology. 

6.2.1 Effect of Ownship Position on Pilots’ Decisions 

In this follow-on experiment, both subject groups had ownship position symbology, but 
only one group of 12 pilots flew with the same size NEXRAD image cell size (small) that 
was used in the previous weather display experiment. Therefore, the effect of having 
ownship position displayed can be examined. Figure 6-1 provides a comparison of the 
frequency of good and poor decisions enroute to Richmond, and figure 6-2 provides a 
comparison of the frequency of good and poor decisions enroute to Wallops Island. 
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Figure 6-1. Effect of Ownship Display on Richmond Decision 



Figure 6-2. Effect of Ownship Display on Wallops Decision 
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As can be seen, the display of ownship position did not have a noticeable effect on the 
frequency of good versus poor decisions. Equally important, the finding that the addition 
of ownship to the display did not have a detrimental effect on the decision-making ability 
of the pilot. Also of interest is the indication that the addition of ownship had a noticeable 
effect on reducing the perceived workload of the weather information display. 

6.2.2 Effect of NEXRAD Image Cell Size on Pilot Decisions 

For the Richmond decision, the data indicates that pilots were more likely to make good 
decisions with the larger NEXRAD image cell size. Figure 6-3 provides a comparison of 
the frequency of good versus poor decisions for the Richmond decision. 


Richmond Decisions 
(with ownship) 



□ Good 
MPoor 


Figure 6-3. Richmond Decisions 


A chi-square significance test was computed to examine differences in observed frequen- 
cies between these groups at the Richmond decision point. The chi-square with one de- 
gree of freedom was 2.73, with a p-value <0. 10. Although this is not regarded as “report- 
ably significant” using the convention that p should be less than 0.05 (i.e. less than a 1 in 
20 chance of the result being a “fluke”), the experimental finding nevertheless remains 
unlikely to be a mere chance outcome (less than a 1 in 10 chance of this). Therefore, the 
chi-square does suggest that the low resolution, large cell size version of the weather dis- 
play may have influenced the Richmond decision. The result is consistent with the pre- 
diction that lower resolution NEXRAD images increase uncertainty over the actual loca- 
tion of hazardous weather. 


For the Wallops Island decision, the data shows that there was not any difference in the 
frequency of good versus poor decisions between the pilots that flew with the larger 
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NEXRAD image cells and those that flew with the small cells. Figure 6-4 provides a 
comparison of the frequency of good versus poor decisions for the Wallops Island deci- 
sion. 


Wallops Island Decisions 
(with ownship) 



Small Cells Large Cells 


Good 

5 5 

Poor 

7 7 


Figure 6-4. Wallops Island Decisions 


As would be expected from a cursory examination of the values in the table, the chi- 
square test was not significant. The two groups were exactly the same in terms of deci- 
sion frequency as defined and analyzed in this experiment. Therefore, NEXRAD image 
cell size did not appear to have an effect on the decisions for the Wallops Island leg. 

It should be noted that some of the poor decisions made by the pilots having the large 
NEXRAD image cells may have been due to pilot procedural issues rather than pilot per- 
ception or weather information issues. There were cases where the pilot made the deci- 
sion to change course to avoid the storms with enough time to avoid them, but delayed 
the turn and drifted too close to the storm. This is reflected in the greater number of mar- 
ginal decisions (coded as either a 2 or 3). 

Table 6-1 provides an overview of the Richmond decisions. A comparison is made be- 
tween the decision results from both experiments and the decision grading criteria. The 
expanded decision criteria scale is applied to the follow-on experiment only. 
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Table 6-1. Overview of Richmond Decisions 


Grading Criteria 

Decision Results 
With Ownship 

Decision Results 
Without Ownship 

Large 

Cells 

Small 

Cells 

Small 

Cells 

No 

Display 

i = 

Strong 

Poor 

Waved off approach 
by tower controller at 
OM, and 

Flew within 5 
miles of a red 
N EXRAD cell 

3 

7 

7 

6 


Poor 

Abandoned approach 
less than 5 miles 
outside OM, but 

— 

— 


Good 

Abandoned approach 
less than 5 miles 
outside OM, and 

Flew more than 
5 miles from a 
red NEXRAD 
cell 

1 

1 

6 

6 


Strong 

Good 

Abandoned approach 
more than 5 miles 
outside OM, and 

8 

4 

Total 

12 

12 

13 

12 


Table 6-2 provides an overview of the Wallops Island decisions. A comparison is made 
between the decision results from both experiments and the decision grading criteria. The 
expanded decision criteria scale is applied to the follow-on experiment only. 

Table 6-2. Overview of Wallops Island Decisions 


Grading Criteria 

Decision Results 
With Ownship 

Decision Results 
Without Ownship 

Large 

Cells 

Small 

Cells 

Small 

Cells 

No 

Display 

1 = 

Strong 

Poor 

While circumvent- 
ing storms using 
own route planning 

Flew within 10 
miles of a red 
NEXRAD cell 

5 

5 

7 

1 


Poor 

While circumvent- 
ing storms, was 
distracted or de- 
layed turn, and 

2 

2 


Good 

Pilot following ATC 
vectors, but 

2 

1 

5 

11 


Strong 

Good 

Pilot avoided red 
cell, and 

Flew more than 
10 miles from a 
red NEXRAD cell 

3 

4 

Total 

12 

12 

12 

12 
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6.2.3 Rationale for Pilots’ Decisions 

The post-flight interview attempted to obtain the pilots’ rationale for their decisions. The 
responses were slightly varied, but centered around three categories and were recorded as 
follows: 


For the Richmond decision, 

• They saw deteriorating weather and decided to divert to Wallops Island. 

• They would try one approach and see how far they could get, but if that 
first approach was not successful, they would divert to Wallops Island. 

• They were concerned that if they did land at Richmond, they could not 
get back out. 

During the interview, the same open-ended question was asked for both decisions, “What 
led you to make the decision to: [insert observed action at decision point]?” 

The results for the Richmond decision are illustrated in Figure 6-5. 



Figure 6-5. Richmond Decision Rationale 
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It appears that for the Richmond decision, a larger number of the subject pilots having the 
large NEXRAD image cells decided not to try an approach and diverted earlier. This 
conclusion is supported with the finding that more pilots having the large NEXRAD im- 
age cells made good decisions than those with the small NEXRAD image cell size. 

For the Wallops Island decision, the same question about rationale was asked and the re- 
sponses were categorized as follows: 

• They saw deteriorating weather and decided to divert around the hazardous area. 

• From their perception, the storms did not look exceedingly bad. 

• They sought advice from ATC and asked for vectors. 

The results for the Wallops Island decision are illustrated in Figure 6-6. 


Wallops Decision Rationale 


□ Saw 

deteriorating 
weather and 
decided to divert 

H Storms didn't 
look that bad 


□ Took ATC advice 
and asked for 
vectors 


Figure 6-6. Wallops Island Decision Rationale 

It appears that in making the Wallops Island decision, the subject pilots having the large 
NEXRAD image cells made better choices by either deciding to change course on their 
own or by seeking the advice of ATC and asking for vectors. Although some of the pilots 
made the decision to divert within plenty of time, their execution of the diversion was in 
some instances delayed and brought them within close proximity to the storms. 

6.2.4 Combination of Decisions from Previous and Current Experiments 

If the frequency of good versus poor decisions is compared across both experiments, it 
appears that for the Richmond decision, the pilots performed better with the weather dis- 
play containing ownship position and large NEXRAD image cells. Figure 6-7 illustrates 
the results of the Richmond decision across both experiments. 
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For the Wallops Island decision, the addition of the weather display significantly reduced 
the ratio of good versus poor decisions. 

Figure 6-8 illustrates the results of the Wallops Island decision across both experiments. 
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Combined Experiments - 
Wallops Island Decisions 




WX Display 

WX Display 

WX Display 

No WX Display 

(no ownship, 

(ownship, 

(ownship, 


sm cells) 

sm cells) 

lg cells) 

11 

5 

5 

5 

1 

7 

7 

7 


Good 

Poor 


□ Good 
■ Poor 


Figure 6-8. Combined Wallops Island Decisions 


6.2.5 Risk Aversion, Weather Knowledge and Experience Analysis 

A hierarchical regression analysis was conducted to examine the possible contribution of 
weather display, weather knowledge, risk aversion, and experience to decision making 
quality as indexed by the four point ratings. Experience was defined as total flight hours, 
instrument hours, and simulation hours since these three variables were very highly cor- 
related {Pearson Product Moment values for r ranged from 0.56 to 0.85). 

Hierarchical regressions were computed for the Richmond decision point, and for the 
Wallops Island decision point. A third was computed taking both decision points into 
consideration. 

6.2.5.1 Richmond Hierarchical Regression Analysis 

The results of the hierarchical regression for the Richmond decision point indicate that 
none of the variables were significant predictors of decision making. Table 6-3 illustrates 
the results of the regression analysis for the Richmond decision. 
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Table 6-3. Richmond Regression Analysis Results 


Dependent Variable = Richmond Decision 


R 

R 2 

Significance 

1 st level: low or high 

NEXRAD image resolution 

.34 

.11 

not 

significant 

2 nd level: weather knowledge 
and risk aversion 

.37 

.14 

not 

significant 

3 rd level: experience 

.49 

.24 

not 

significant 


6.2.5.2 Wallops Island Hierarchical Regression Analysis 

The results of the hierarchical regression for the Wallops Island decision point indicate 
that the greatest influence on decision quality came from weather knowledge. Not sur- 
prisingly, there was no significant effect attributable to the weather display, since the pi- 
lots had exactly the same decision frequencies whether they used a low or high resolution 
display. Table 6-4 illustrates the results of the regression analysis for the Wallops Island 
decision. 


Table 6-4. Wallops Island Regression Analysis Results 


Dependent Variable = Wallops Decision 


R 

R 2 

Significance 

1 st level: low or high 

NEXRAD image resolution 

.00 

.00 

not 

significant 

2 nd level: weather knowledge 
and risk aversion 

.55 

.30 

significant 

p =. 06 

3 rd level: experience 

.64 

.41 

not 

significant 
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The chi-square test suggested that better decision making may have occurred at the 
Richmond decision point when a low resolution (and, therefore, we posit, high uncer- 
tainty) weather display was in use. However, the regression analyses do not help to ex- 
plain why such a difference might have occurred. Hierarchical regression results for 
Richmond included no significant predictors of rated decision making. It cannot be said 
that the probable difference in decision making (albeit at p< 0.1) was due to variability in 
pilots’ weather knowledge, their risk aversion, or flight experience levels. The regression 
results for the Wallops Island decision do suggest the importance of weather knowledge 
and risk aversiveness. 


6.2.5.3 Combined Hierarchical Regression Analysis 

Considering the influence of weather display, weather knowledge, risk aversion, and ex- 
perience on the combination of decisions at Richmond and Wallops Island together sug- 
gests that weather knowledge may be in part responsible for decision quality as rated in 
this experiment. The p-value associated with weather knowledge and risk aversion was 
less than 0.1, and the significant t- value associated with weather knowledge was the 
source of the significant effect (rather than risk aversion). This seems to suggest that 
weather knowledge as measured by the declarative knowledge test may have influenced 
decision quality at both decision points. Table 6-5 illustrates the results of the combined 
Richmond and Wallops regression analysis. 


Table 6-5. Combined Richmond and Wallops Island Regression Results 


Dependent Variable = 

Ratings of decision making, both locations 


R 

R 2 

Significance 

1 st level: low or high 

NEXRAD image resolution 

.14 

.02 

not 

significant 

2 nd level: weather knowledge 
and risk aversion 

.52 

.27 

significant 
p=. 09 

3 rd level: experience 

.58 

.34 

not 

significant 


6.3 Qualitative Assessments 

The difficulties in the use of the weather information display appear to include workload 
problems, incorrect assumptions about the relative location and age of displayed weather 
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data, inadequate correlation to other weather information sources, inability to interpret 
convective weather movement and insufficiency of training. 

While it was required that all candidate subject pilots for this experiment be qualified and 
current as instrument pilots, the 24 pilots ultimately selected to participate demonstrated a 
very wide range of proficiency in instrument flight. The pilots’ experience ranged from 
66 hours of instrument time to over 2,000 hours of estimated actual instrument time. 

Their proficiency in instrument flight operations was very probably quite representative 
of the population of general aviation pilots having similar qualifications and levels of ex- 
perience. 

6.3.1 Workload Issues 

This experiment was not designed to specifically measure or quantify workload in rela- 
tion to the use of the weather information display. However, the observers made general 
observations relating to workload, and the pilots were asked a series of questions relating 
to their perception of the workload. 

Two areas of workload were investigated including, 1) the overall workload for the entire 
mission, and 2) the workload specific to the use of the weather information display. 

6.3.1. 1 Overall Mission Workload 

The observers gathered workload cues such as incorrect or inappropriate procedures, 
hesitation in communication, changes in voice pitch, control excursions, flight technical 
errors, procedural hesitation, navigation errors, haphazard search techniques, autopilot 
use, training transfer problems and physiological indications gained from the interview. 

The subject pilots were asked to comment on the workload for the entire mission. During 
the post-flight interview, the pilots were asked, “Did you feel ahead of the airplane?” The 
results are illustrated in Figure 6-9. 
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Perceived Comfort Level 



Figure 6-9. Pilots’ Perceived Workload for Entire Mission 


The pilots’ perceived workload closely matched the observations and reflected the in- 
tended pace of the experiment — to explore the effects of the weather display in a high- 
workload and realistic flight environment. Another indicator of the workload is the 
amount of autopilot use. 

The pilots had an autopilot available during the experiment and were trained on its use 
during the training session. During the pre-mission briefing, the pilots were instructed to 
use the autopilot if they felt it necessary to do so, but there was neither any requirement 
nor penalty in its use. Figure 6-10 illustrates the extent to which the pilots used the auto- 
pilot in the experiment. 


AP usage during flight 

100 . 0 % 

80 . 0 % 

60 . 0 % 

40 . 0 % 

20 . 0 % 

0 . 0 % 


Figure 6-10. Percent of Flight Time Autopilot Used 
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Even though the autopilot was used for an average of 85 percent of the time in flight 
across all the subject pilots, some pilots were too busy to effectively integrate the use of 
the weather display into their procedures. Others were able to effectively use only one or 
two functions of the display. All of the pilots in both groups stated the autopilot was ei- 
ther essential to the safe accomplishment of the flight or substantially reduced the work- 
load of flying in instrument conditions. Nearly all of the pilots stated that in reducing 
their workload, the autopilot made it possible for them to make more effective use of the 
weather information display. 

Most of the pilots said they felt comfortable with the autopilot and relied on its use to re- 
duce the workload. The autopilot was, on average, in use for 85 percent of the flight (for 
all pilots). In the previous weather display experiment, the autopilot was in use 83 percent 
of the flight, on average. Many of the pilots stated that without the autopilot, they would 
have succumbed to an early termination of the flight. 

Because the RTI Simulation Facility is not a full-motion/full-reactive type of simulator, 
the lack of a peripheral visual system, motion base and reactive flight controls more than 
likely contributed to the extensive use of the autopilot. Without the normal visual and tac- 
tile sensations, hand flying this simulator would be difficult in a high workload situation. 
Therefore, the high use rate of the autopilot is fully understandable. Additionally, from an 
experimentation standpoint, the consistent use of the autopilot reduced a significant vari- 
able — flight technical error. 


6.3.1. 2 Workload in use of Weather Display 

During the course of this experiment, subjective measures were undertaken to determine 
the pilots’ perception of their workload in using the weather display. 

On the final Weather Display Questionnaire (Appendix N), the pilots were specifically 
asked how the weather display affected their workload. The same question was also 
asked of the pilots in the previous experiment that flew with the weather display without 
ownship position symbology. The results are illustrated in Figure 6-11. 
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Weather Display Workload 
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Figure 6-11. Pilots’ Perception of Weather Display Workload 


The number of pilots that felt the weather display decreased their workload steadily in- 
creased as ownship position and larger NEXRAD image cell features were added. The 
surprising number of pilots that felt the weather display increased their workload [in the 
no ownship group of the previous experiment] is most likely attributable to the lack of a 
display of ownship position and the resulting difficulty in determining their position on 
the display. Many responses to the specific workload question cited that, “... trying to 
find my position on the display increased my workload.” 


6.3.2 Use of Available Weather Data Sources 

The pilots were able to access the normal in-flight weather services through the simulator 
radio, including: 

• FSS - Flight Service Station 

• ATC - Air Traffic Control (tower, departure and approach) 

• FW - Flight Watch 

• ATIS - Automatic Terminal Information Service 

• ASOS - Automated Surface Observation System 

The type and frequency of weather information sources used by the pilots were recorded 
on the Observer Form (Appendix K). 
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6.3.2.1 Richmond Area Weather Sources 

A breakdown of the weather information sources used, in addition to the weather infor- 
mation display, by the subject pilots for the Richmond leg is illustrated in Figure 6-12. 


Richmond Decision - 
Weather Information Sources Used 


Text METAR 
RIC ATIS 
RIC ATC 
Flight Watch 
FSS 

0 2 4 6 8 10 12 14 



B Large Cells 
■ Small Cells 


Amount of Responses 


Figure 6-12. Additional Weather Sources Used for the Richmond Decision 


The primary source of weather, for the Richmond decision, was reported by the pilots 
using the small-cell display to be the weather information display itself, and by the pilots 
using the large-cell display to be ATC (Figure 6-13). None of the pilots reported Flight 
Watch or Flight Service to be their primary weather source. 
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Richmond Decision - Primary Source 
of Weather Information 

Combination 
Preflight 
FISDL Display 
ATC 
ATIS 

0 2 4 6 8 

Amount of Responses 






r 

3 



H Large Cells 
■ Small Cells 


Figure 6-13. Primary Source of Weather Information for Richmond Decision 


When asked in the post-flight debriefing about their use of Flight Service Stations, some 
pilots commented that they do not use the service much — due mostly to the difficulty in 
interpreting the verbal information and the excessive amount of time necessary to collect 
the information. Additionally, the Flight Service does not generally know their specific 
location and obtaining specific route information is difficult. While en-route to Rich- 
mond, 2 participants in the small-cell group contacted Flight Service before departure, 
and 2 contacted Flight Watch while in the air. One participant in the large-cell group con- 
tacted Flight Watch before departure. 
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6. 3. 2. 2 Wallops Island Weather Sources 

A breakdown of the weather information sources used, in addition to the weather infor- 
mation display, for the Wallops Island leg is illustrated in Figure 6-14. 


Wallops Island Decision - Weather 
Information Sources Used 

(in addition to the weather display) 

Text METAR 
WAL ASOS 
Flight Watch 
FSS 
ATC 

0 2 4 6 8 10 12 14 

Amount of Responses 

Figure 6-14. Additional Weather Sources for the Wallops Island Decision 



For the Wallops Island decision, both groups reported that the weather information dis- 
play was the primary source of weather information (Figure 6-15). Additionally, both 
groups reported ATC as the secondary source of weather. Flight Service/Flight Watch 
was not reported as a secondary source. While en-route to Wallops Island, 1 participant in 
the small-cell group contacted Flight Service and 6 participants contacted Flight Watch. 

In the large-cell group, 2 participants contacted Flight Watch. 
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Wallops Island Decision - Primary 
Source of Weather Information 


FW/FSS 

Text METAR 

FISDL 

Display 

ATC 


0 2 4 6 8 10 

Amount of Responses 



Figure 6-15. Primary Source of Weather for the Wallops Island Decision 


ATC was considered to be secondary to the weather display as the source of weather in- 
formation by both groups en-route to Wallops Island. Only one pilot reported Flight Ser- 
vice/Flight Watch to be the primary source of weather information for the Wallops Island 
decision. This indicates a need for pilot training in the appropriate use of FISDL weather 
data as an aid to collaborative decision making with appropriate weather service provid- 
ers. 


6.3.3 Interpretation of the Weather Information Display 

Nearly all of the subject pilots were enthusiastic about the potential of the weather infor- 
mation display for improving their awareness of weather conditions, but many misinter- 
preted or did not access the information available from the display. Many of the poor 
weather decisions were made because the pilots were not aware of the deteriorating 
weather conditions and were consequently surprised that they could not get into the 
Richmond airport or that there were thunderstorms over the bay between Richmond and 
the Wallops Island airport. Nearly all recognized that a thunderstorm was in the vicinity 
of the Richmond airport — most failed to correctly recognize the level of the hazard this 
presented to them. 

When those who did so were asked why they attempted to proceed with the approach past 
the outer marker, they said they had not recognized how close they were getting to the 
hazardous areas of the storm, or that it did not look that bad. Many said they had decided 
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to proceed with one approach into the Richmond airport and “see what happens.” Most of 
these pilots said they were looking for other clues as to the severity of the weather such 
as turbulence levels, lightning or rain intensity. 

6.3.3.1 Judging Proximity to Hazardous Weather 

Many pilots had difficulty correctly determining their position in relation to the storm 
cells. The weather display did not employ any type of range determination graphics 
around the ownship icon. The only assistance provided for range determination was a 
map distance scale in the upper right comer of the display window. 

The difficulty in determining distance to the hazardous convective activity was reflected 
in the pilots’ comments. Many pilots suggested that some form of range rings or other 
similar indication should be available to aid in estimating distances. The comments re- 
ceived from the pilots during the interview confirmed that they welcomed the ownship 
position icon and found it accurate and easy to use. The pilot comments indicated that 
they trusted the display elements to be accurate relative to each other, but were not as 
confident that the ownship position was an accurate representation of their track over the 
ground. 


Question 13 . The pilots were asked of the positional accuracy of the ownship position in 
relation to storm information. 

In using the weather display, I felt that I generally knew the aircraft posi- 
tion relative to any storms. 

(mean score of 4.58, standard deviation of 0.72) 

[score of 5 = agree, score of 1 = disagree] 


6.3.3.2 Situational Awareness 

On the final Weather Display Questionnaire (Appendix N), the pilots were asked how the 
weather display affected their situational awareness. 

Did you find that the weather display increased or decreased your 

situational awareness? 

All of the pilots, except one, answered this question by saying that the weather display 
increased their situational awareness. The pilot that found the display decreasing his situ- 
ational awareness was a very low time pilot that was already overloaded by the demands 
of the mission scenario. Many of the pilots commented that the weather display slightly 
increased their workload, but gave them a vast improvement in situational awareness. 


61 



6. 3. 3. 3 Recognizing and Interpreting Effects of Delay 

The pilots were briefed twice (during the introduction and again during the familiariza- 
tion flight) that the NEXRAD image could be from 7 to 14 minutes old and to check the 
image timestamp with the onboard clock. They were also apprised that the METAR in- 
formation could be as much as an hour old and to check the issue time. Due to the lack of 
an ocular eye tracker, empirical data for how the subject pilots used the timestamp infor- 
mation was not available. Insight into how they used the timestamp information was only 
available through the interview process. 

During the post-flight interview, some of the pilots indicated that they were aware that 
the NEXRAD images were up to 14 minutes old, and either noted the amount of latency 
or just assumed a fixed delay. Others stated that 14 minutes old is real-time, compared to 
preflight weather charts that could be over an hour old. 

In response to the question “How did you determine the age of the weather information?” 
in the Weather Display Questionnaire (Appendix N), 14 pilots responded that they used 
the time-stamp information to determine the age of the weather information (NEXRAD 
and METAR). Six (6) pilots responded that they just assumed a fixed delay of informa- 
tion — most likely due to the increased cognitive load required to mentally subtract the 
current time from the time-stamp. Three pilots did not determine the age of the weather 
information. 


6. 3.3.4 NEXRAD Image Cell Size Effects on Richmond Decision 

In the analysis of the Richmond decision, the data shows that a greater number of good 
versus poor decisions were made with the introduction of large NEXRAD image cells. 
One explanation for this finding converges on the stimulus area effect. The stimulus area 
effect states that the larger the visual area of a warning stimulus, the greater importance it 
holds. 

The implementation of automobile brake lights in the rear window decreased by half the 
rear-end accident rate (Malone, 1986). The addition of a greater amount of “warning” 
area contributed to the greater importance of the warning and caused drivers to brake ear- 
lier. 

This extensively documented stimulus area effect is a possible explanation for the ex- 
periment results. With the larger cell sizes, fewer pilots flew near the storms. The “stimu- 
lus area” effect states that the bigger the stimulus, the more important it becomes. So a 
bigger red cell would be more ominous and a pilot would keep farther away. This larger 
stimulus area also created a greater uncertainty in the exact location of the hazardous 
weather, which led the pilots to select a track farther away from the depicted weather. A 
comparison of NEXRAD image cell sizes is illustrated in Figure 6-16. 
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1914Z NEXRAD Image 1914Z NEXRAD Image 

Small Cells (4 km sides) Large Cells (8 km sides) 


Figure 6-16. Comparison of Small versus Large NEXRAD Cells 


An unlikely connection may be made between NEXRAD display cell size and U.K. traf- 
fic research. In the U.K., it was found that drivers disliked new, much smaller traffic is- 
lands (roundabouts) at multiple road junctions. Drivers were unsure of the optimal track 
to take across the (effectively widened) junction. Accidents, however, were significantly 
reduced. The drivers’ uncertainty, the researchers concluded, created greater attention to 
clearances, tracks through the junction and the behavior of other drivers. It is instructive 
to note the way in which users’ subjective opinions and objective performance dissociate 
in this case (as in many cases), and how uncertainty can lead to improved decision mak- 
ing. 

The relevance of this to the present experiment flows from the analogy between weather 
display cell size (i.e. resolution) and traffic island size, and the role of uncertainty. There 
is reason to believe that lower resolution weather images must, in their nature, increase 
uncertainty over the “actual” location of weather boundaries and edges. If so, perhaps as 
in the road research case, this uncertainty translates into more data seeking and cognitive 
processing in this case of weather and related aspects of the flight. The lower resolution 
may have accomplished the goal of data-linked strategic weather avoidance, as the sys- 
tem was initially envisioned. 


6.3.4 Retrospective Pilot Comments 

In the course of the post-mission interview, the pilots were asked what their actions 
would have been if the weather display was not available. This was an open-ended ques- 
tion and purposefully placed at the end to allow the pilots to become familiar with the 
tone of the interview and become comfortable with the intended level of introspection. 
There were two parts to the questioning, the Richmond leg and the Wallops Island leg. 
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The pilots responded to the question, “What would you have done if you did not have 
the weather display... 


...at Richmond?” 

“I would’ve taken longer to break off at Richmond.” 

“Would 1 ve diverted later into Richmond, and been stuck on ground.” 

“Would’ve talked to ATC more and ask positional questions.” 

“Would rely on ATC more, but without this display, probably wouldn’t have 
done much different.” 

“Would have used Flight Watch more and probably not even attempted 
Richmond.” 

“...not even tried approach into Richmond. Would have diverted sooner.” 
“Would have made same decisions, but this display would be great back-up.” 
“Would not have even gone to Richmond.” 

“Would have heavily taxed ATC to help pick way through weather.” 

“Probably would have just returned to Newport News. ATC radar not that 
accurate, so would not have attempted flight with thunderstorms in the area.” 

“Would have treated Richmond the same.” 

“Would have diverted to Wallops quicker. The display led me into Rich- 
mond. Value of display increases with leg lengths. Just too quick on short 
flights to look at trends.” 

“Would not have attempted approach into Richmond and would have talked 
to ATC more and further out.” 

“Would have flown in worse conditions without knowing, and would have 
had to rely more on Flight Service and ATC.” 

“Without the display, would have gone direct to Wallops from Newport 
News.” 
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...at Wallops Island?” 


“I would’ve asked for more help from ATC and Flight Watch.” 

“Would’ve pushed it less into Wallops.” 

“Without display, would have asked for more vectors and let ATC help in 
routing.” 

“Probably would have flown right into storms over bay. Without display, 
could not have made turns as soon ... but at Wallops, would have flown into 
storm before asking for help. Used display to slow down before penetrating 
weather.” 

“Would have been more likely to go farther south around Wallops storm and 
avoid known weather by greater margin.” 

“Would have been a little more nervous going around Wallops weather.” 

“Would’ve probably penetrated Wallops storm, turned around, and ask for 
hold. Using the weather display allowed me to circumnavigate the storms — 
without it the flight would have been longer and at the mercy of ATC.” 

“...more dependence on ATC and Flight Watch for vectors around weather.” 

“Without display, would have flown into the storms at Wallops.” 

In review of these comments, the majority of the pilots were greatly influenced by the 
weather display, and without the display most would have avoided the hazardous weather 
by a greater margin. Although these are just subjective comments, the data from the Wal- 
lops decision supports the comments. 

The following two questions were asked on the Immediate Reactions Questionnaire, and 
support the pilot comments as to the reliance on the weather display. 

Question 3 . This question explored the pilots’ perception of the degree to which they felt 
they were able to interpret the weather information display and the extent to which the 
display influenced their decision making. 

I attribute much of my decision making to my interpretation of the 
Weather Display. 

(mean score of 4.54, standard deviation of 0.88) 

The mean score of 4.54 (a 5 being “totally agree ”), combined with a small deviation 
(SD=0.88), indicates that the weather information display played a large part in the de- 
cision process during the flight. 
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Question 8 . When asked about the validity of the weather information display, the reac- 
tions consistently held the display validity in high regard. 


The degree of validity of the weather data appearing on-screen was a 
factor I felt I held in mind as I flew. 

(mean score of 4.58, standard deviation of 0. 72) 
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7 Conclusions 

The objective of this follow-on experiment was to investigate the effects of ownship posi- 
tion information and NEXRAD image cell size resolution on pilot decision making. The 
experiment was purposefully designed to determine how a pilot could misuse the weather 
information display and to identify the areas that need further design or regulatory con- 
sideration. 

The first hypothesis stated that: the introduction of ownship position information on the 
weather display will improve navigation decisions. 

For either the Richmond or Wallops decisions, the introduction of ownship position in- 
formation did not improve decisions. Therefore, the hypothesis is rejected. It was noted 
that, 1) the addition of ownship to the display did not have a detrimental effect on the de- 
cision-making ability of the pilot, 2) that the addition of ownship symbology reduced the 
workload associated with use of the weather information display. 

The second hypothesis stated that: NEXRAD image resolution will impact navigation 
decisions. 

For the Richmond decision, the reduction of NEXRAD image cell size resolution had a 
positive, although not statistically significant, effect on decision making. For the Wallops 
decision, the reduction of NEXRAD image cell size resolution did not have an effect on 
decision making. 

It is anticipated that with sufficient training, careful use of the cockpit weather display, 
and prudent pilot procedures in instrument flight conditions, the emerging cockpit 
weather display products will provide substantial improvements to the safety of flight. It 
is important to note that this experiment, like the one that preceded it, was designed spe- 
cifically to identify potential hazards in the use of cockpit weather displays. 

Every aspect of the design of this experiment was undertaken with this objective in mind, 
including subject pilot selection, subject pilot training, and the mission scenario. Pilots 
were selected so as to provide as wide and representative a range as possible of the ex- 
perience, knowledge of weather and risk aversion of the population of general aviation 
pilots who might use these emerging cockpit weather display products. The training pro- 
vided the subject pilots was tailored so as to provide them with sufficient familiarity with 
the experimental equipment to successfully accomplish the mission scenario, while at the 
same time creating a reasonable probability that within the population of subject pilots 
selected, potential hazards in the use of the equipment might become apparent. Likewise, 
the mission scenario incorporated in the experiment was selected to ensure that it could 
be accomplished by the average pilot with careful attention by the subject pilot to the in- 
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strument flight procedures, but offered sufficient opportunity for observation of human 
error in the use of the prototype cockpit weather display where such hazards might exist. 

The significant issues addressed in the experiment are summarized below. 

7.1 Weather Information Display Interpretation Issues 

The pilots’ use of the weather display did not demonstrably improve the decision out- 
comes, as defined in this study. A noticeable improvement in decisions on the Richmond 
leg was only seen when large NEXRAD cells were implemented on the weather display. 
For the Wallops Island decision, the use of the cockpit weather display increased the 
number of poor decisions independent of the NEXRAD image cell size. 

Causes for these findings include difficulty in position determination, short leg duration, 
NEXRAD image cell resolution and workload issues. 

7.1.1 Position Determination 

The implementation of ownship symbology on the display did not have an outwardly no- 
ticeable effect on the frequency of good versus poor decisions. However, the introduction 
of ownship had a markedly positive effect on reducing the perceived workload. This was 
more than likely a result of the reduced cognitive workload required to determine the air- 
craft position in relation to the hazardous convective activity shown on the weather dis- 
play. 

In the previous weather display experiment, the pilots were presented with a weather dis- 
play that did not include the ownship symbology. During the interview process, the pi- 
lots’ primary complaint was the lack of ownship symbology. With the addition of own- 
ship symbology in this follow-on experiment, the pilots welcomed the ownship symbol- 
ogy and felt comfortable with its use. The experiment did not show any drawbacks to the 
implementation of ownship symbology on the display. 

7.1.2 NEXRAD Image Cell Resolution 

The introduction of larger NEXRAD image cells had a positive, although not statistically 
significant, effect on decision making for the Richmond leg of the scenario. 

The introduction of larger NEXRAD image cells did not have an effect on decision mak- 
ing for the Wallops Island decision. The difference may be due to procedural issues 
rather than perceptual issues. There were many cases en-route to Wallops Island where 
the pilot made the decision to change course to avoid the storms with enough time to 
avoid them, but delayed the turn and drifted too close to the storm. The data suggests that 
the higher fidelity of the NEXRAD image caused more workload as pilots tned to cut 
comers more closely to circumnavigate the weather hazards, and then had to adjust 
course due to the latency of the image. 
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The nature of the Richmond decision was more procedure-based, i.e. in an instrument 
approach, whereas specific decisions are made at specific points. The display using larger 
NEXRAD image cells contributed in a positive way to those decisions. 

The nature of the Wallops Island decision was not so much a procedure-based decision. 
Even though the AIM specifies a 20-mile separation from convective weather, actual op- 
erations do not treat this buffer as an absolute. There are just too many variables with 
convective weather activity to make a strictly procedure-based decision. Some of those 
variables are, thunderstorm lifecycle stage, wind direction, altitude, aircraft capabilities 
and passenger comfort. 

7.1.3 Workload Reduction 

The number of pilots that felt the weather display reduced their workload increased as 
ownship symbology was implemented, and increased further when larger NEXRAD im- 
age cells were implemented. 

In the previous weather display experiment (without ownship symbology), a large num- 
ber of pilots felt that the weather display increased their workload. This is most likely at- 
tributable to difficulty in determining their position on the display and the relationship of 
their aircraft to hazardous weather. The inclusion of ownship symbology decreased per- 
ceived — as well as observed — workload, which allowed more time for the pilots to attend 
to other flight duties. 

7.2 Weather Source Information Issues 

The display of NEXRAD mosaic images substantially increased the pilots’ awareness of 
the general location of convective weather in their vicinity. The attractive visual display 
of these images, however, caused some pilots to depend too heavily on the weather in- 
formation display for their information regarding hazardous convective weather. As a re- 
sult, they did not feel it was necessary to obtain additional and corroborating information 
from other available sources. This indicates a need for pilot training in the appropriate 
use of a cockpit weather display as an aid to collaborative decision making with appro- 
priate weather service providers (Flight Service/Flight Watch). 

7.3 METAR Issues 

Three METAR issues were observed during the experiment, including, latency, coding 
and graphical METAR use. 

7.3.1 METAR Coding 

The METAR textual information was presented in typical ICAO teletype codes, and the 
experiment found that the interpretation of the codes in a high workload environment is 
prone to errors. In this experiment, many of the pilots had difficulty interpreting the 
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codes. Many errors were observed and excessive fixation times were observed when the 
pilots attempted to decode the METAR information. 

It was noted in this experiment, and in the previous experiment that the airline pilots who 
participated did not have as much difficulty in interpreting the reports as the general avia- 
tion pilots did. This was undoubtedly due to the increased use of coded reports by airline 
pilots. Many of the pilots commented that the METARs would be more useful if they 
were displayed with their English translation, much as DU ATS provides the English 
translation. 

7.3.2 METAR Latency 

In many cases the METAR report timestamps indicated that they were up to an hour old, 
even if the conditions were unchanged and the data was still current. This apparent la- 
tency caused many of the pilots to disregard the information in favor of ATC or pilot re- 
ports. Some of the pilots even kept the METAR display page selected during the ap- 
proach into Richmond while waiting for an update that reflected the current conditions. 
The perceived latency of METAR data caused many pilots to disregard the information. 

7.3.3 Graphical METARs 

The pilots did not use the graphical METARs to a great extent, either because of their 
limited information or inability to present timely information. Many of the pilots reported 
that the information contained in the graphical METAR symbology was not complete and 
required that the textual METARs also be consulted. The graphical METARs were in- 
tended to present a general synopsis of surface conditions, but because of their latency 
and limited information, they fall short of their intended purpose. 

7.4 Judging Proximity and Movement of Hazardous Weather 

The age of the NEXRAD images on the weather display led to noticeable errors commit- 
ted by many of the pilots in the course of determining the proximity and rate of move- 
ment of the hazardous convective weather. 


7.5 Stimulus Area Effect 

Analysis of the Richmond decision indicated that better decisions were made with the 
introduction of larger NEXRAD image cells. An explanation for this finding converges 
on the stimulus area effect. The stimulus area effect states that the larger the visual area 
of a warning stimulus, the greater importance it holds. 

The larger stimulus area — presented by the larger NEXRAD image cell size — created a 
greater uncertainty in the exact location of the hazardous weather, which led the pilots to 
select a track farther away from the depicted weather. 
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8 Recommendations 


8.1 AIM and Advisory Circular Recommendations 

The following passage is repeated from the previous weather information display ex- 
periment. The findings of the investigation of ownship and NEXRAD image cell size is- 
sues undertaken in this experiment do not change the recommendations of the previous 
experiment for the AIM and Advisory Circulars. 

The depiction of weather information in the cockpit, including NEXRAD and 
METAR products, will be delayed due to the time required for the collection 
and distribution of vast amounts of weather information available. 

The time required producing the NEXRAD mosaic image on the display in- 
cludes a six-minute cycle for the individual NEXRAD radars to scan and ob- 
serve the data. An additional interval is required for the automated processing 
of the NEXRAD data necessary to merge all the individual NEXRAD radar 
images. A further delay is introduced into all the FISDL products by a broad- 
cast transmission cycle delay. The communication architecture of the FISDL 
broadcast will determine the magnitude of that delay for any specific FISDL 
product. 

It is essential that the pilot become fully proficient in determining and main- 
taining a comprehensive awareness of the age of each of the FISDL display 
weather information products. Thus, the pilot would be able to effectively and 
accurately integrate this information (NEXRAD image time stamps, METAR 
text time data, etc.) with the information gathered from the other sources. 

Because of the inherent production delays, the weather information provided 
by the FISDL display should not be used for avoiding hazardous weather in a 
tactical manner, such as finding one’s way through a line of thunderstorms. In 
the time that it takes for a NEXRAD image to be produced and transmitted, a 
storm cell could have moved a significant distance. Stonn cells can also some- 
times develop very quickly to hazardous levels within the update time of 
NEXRAD images. Therefore, NEXRAD images should be used in the more 
strategic sense to avoid areas of convective activity by a wide margin. 

Weather information provided by the FISDL display in text form (METAR, 
TAF, etc.) could be up to an hour old, and should only be used for gathering 
an understanding of weather conditions over a large geographical area. Other 
independent sources of information (ATIS, ASOS, FSS, Flight Watch, ATC) 
must also be used in conjunction with the FISDL display to assure a complete 
understanding of the weather condition. 
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Pilots should be fully aware that the FISDL display does not contain sufficient 
information to support navigation, and it should not be used as a replacement 
for any aspect of approved navigation procedures and equipment. While the 
FISDL display can increase the pilot’s situational awareness, particularly with 
respect to weather conditions, the display cannot be successfully used to de- 
termine headings, direction, or distances with the accuracies and reliability 
that are required for navigation. 

8.2 Recommendations for Weather Display Manufacturers 

Additional recommendations are provided for the consideration of the cockpit weather 
display system manufacturers and certification authorities. 

8.2.1 Provide Ownship Position 

The inclusion of ownship position symbology on a weather information display had a 
marked improvement in lowering workload and did not have a detrimental effect on deci- 
sion making. With the proliferation of moving map displays in modem cockpits, pilots 
are becoming accustomed to seeing ownship symbology that they use to determine their 
position on the map display. 

The benefits of ownship symbology appear to outweigh the concerns associated with the 
display of real-time position information and old information (7-minute-old NEXRAD 
image) on the same display. 

8.2.2 Provide Direction and Rate of Weather Motion 

Many of the pilots in this experiment had difficulty determining the movement of the 
convective weather and asked for either a “looping” capability (playback of preceding 
images) or vector arrows showing speed and direction (similar to the National Weather 
Service radar depiction charts). “Looping,” however, may cause an increase in workload 
while a pilot attempts to predict the future track of a storm from past data. Further inves- 
tigations need to be undertaken to determine the effects of looping on pilot performance 
and workload. 

8.2.3 Provide for Intuitive Distance Determination 

Many of the pilots in this experiment made poor estimations of the distance between the 
aircraft and the convective weather. This misperception was a significant contributor to 
the inability of many of the pilots to effectively use the weather information display. This 
indicates that there is a need to provide an intuitive method to determine range informa- 
tion on a weather display. 
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8.2.4 Provide for Intuitive NEXRAD Image Age 

There is a concern that the display of stale weather information in the cockpit may cause 
interpretation difficulties and lead to use of stale weather information. This rationale was 
due to the difficulty and cognitive processes required in subtracting the current time from 
the NEXRAD image timestamp and predicting the weather movement. The problem will 
only be exacerbated should manufacturers choose to offer NEXRAD images on a “pay- 
for-view” basis with an even greater interval, for some users, between images. A more 
intuitive method needs to be implemented that alleviates the mental calculations neces- 
sary in correctly determining the age of NEXRAD/METAR data. 


8.2.5 Provide METAR Code Translation 

In this experiment, the pilots’ commented that the METAR reports were difficult to inter- 
pret, took too much time and were not of much use because they were old. Currently, the 
textual METAR information is presented in typical teletype codes and although this is the 
information that will be broadcast at no charge to the user, the interpretation of those 
codes in a high workload environment causes many errors. In this experiment, many er- 
rors were observed and excessive time was devoted to decoding the METAR informa- 
tion. Similar findings were found in the previous weather display experiment. This indi- 
cates a necessity for METAR code translation. 

Additionally, what pilots need prior to commencing an approach is the current official 
observation (via data-link) for the airport (METAR or SPECI). Rapidly changing control- 
ling elements such as R.VR are best provided (in the near term) directly from the 
TRACON or tower controller who has direct readouts of the current conditions. In the 
future, consideration should be given to the provision (via data link) of direct readouts of 
current conditions, such as ASOS, to the pilot. 
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9 Recommendations for Further Research 

Proposed research topics fall into four broad categories: evaluation of cell movement de- 
pictions, integration and display of new weather products, optimum NEXRAD image cell 
size and development of a training system for aircraft cockpit weather information man- 
agement. 

9.1 Conduct Evaluations of Cell Movement Depictions 

Many of the weather display issues are centered around the lack of NEXRAD image cell 
movement depiction. Many pilot comments confirmed this experimental finding and sug- 
gested two methods to depict cell movement, including looping and vector arrow icons. 

A follow-on experiment would investigate the effects on pilot performance of adding al- 
ternative concepts for depicting movement of hazardous convective weather. 

9.2 Develop Concepts for Display of Predictive Weather Products 

Comments of the pilots in this experiment identified concern about the lack of lightning 
data, and lack of accurate information about the movement of storms. The combination of 
NEXRAD data and lightning data already exists in the National Convective Weather 
Forecast (NCWF) product created by the Aviation Digital Data Service. This image con- 
tains precipitation data that is filtered, processed and combined with lightning data to 
provide a near real-time depiction of the predicted movement of thunderstorms. 

An experiment could be developed to investigate the effects on pilot performance of this 
combined forecast, and techniques for implementation could be developed. 

9.3 Determine Optimum NEXRAD Image Cell Resolution 

This experiment observed that the increase of NEXRAD image cell size from 4km to 
8km had a positive effect on the decision making ability of the pilot, but increasing the 
size of NEXRAD image data even further can not guarantee further improvement in pilot 
decision making. A point of diminishing returns may be reached if cell size is further in- 
creased. If the NEXRAD image cells become too large, a pilot would not trust the 
weather depiction when the cockpit display does not correlate with the view out the win- 
dow. An experiment could be developed to explore the relationship of increasing 
NEXRAD image cell size with decision making and to determine the point of diminish- 
ing returns. 

9.4 Develop Training for Weather Information Displays 

A training curriculum should be developed to support the implementation and proper use 
of weather displays in the cockpit. The curriculum needs to include appropriate manuals 
and modem interactive multi-media training techniques that would highlight common 
mistakes and improper usage of the weather display information, and develop appropriate 
operational procedures for the use of weather display systems. 
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Appendix A. Flight Information Services Description 

System Overview 

The Flight Information Services (FIS) Data Link Display will provide pilots with the dis- 
play of certain aeronautical weather and flight operational information. This information 
will be displayed using both text and graphic formats. Service providers will provide a 
broadcast FIS system using VHF data link. This system will provide coverage throughout 
the Continental United States from 5000 feet AGL to 17,500 feet MSL, except in those 
areas where this is unfeasible due to mountainous terrain. Aircraft equipment will include 
at least an appropriate receiver and display unit. This system will provide, free of charge, 
the following Basic Products: 

Aviation Routine Weather Reports (METARs), 

Special Aviation Reports (SPECIs), 

Terminal Area Forecasts (TAFs), and their amendments, 

Significant Meteorological Information (SIGMETs), 

Convective SIGMETs, 

Airman’s Meteorological Information (AIRMETs), 

Pilot Reports (both urgent and routine) (PIREPs), and 

Severe Weather Forecast Alerts (AWWs) issued by the FAA or NWS. 

Additional products, called Value Added Products, will be available from the FIS provid- 
ers on a paid subscription basis. Most of the value-added products are expected to be 
graphical in nature and may include but are not limited to: 

National, Regional and Local NEXRAD mosaics 

Icing forecasts 

Turbulence forecasts 

Graphical METARs 

Winds 

Cloud Tops 

The FIS products will be required to conform to FAA/NWS standards. Specifically, the 
FIS weather information must meet the following criteria: 

1. The products are either FAA/NWS accepted aviation weather products, or 
based on FAA/NWS accepted weather products. 

2. In the case of a product which is the result of the application of a process 
which alters the form, function or content of the base FAA/NWS accepted 
weather product(s), that process must be: 

a) An established, conventional aviation weather process used in stan- 

dard U.S. aviation weather information systems, and, 

b) Managed by a qualified aviation meteorologist. 
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3. National Airspace System (NAS) status products (such as NOT AMs, Spe- 
cial Use Airspace Status, etc.) will include verbatim transmissions of FAA 
products. If graphics are used to describe NAS status, the basic text prod- 
uct will be readily available to the pilot for reference. 

Operations 

To receive FIS broadcasts, an aircraft must have a data link radio and appropriate display. 
Both of the initial FIS service providers were awarded frequencies between 136.425 MHz 
and 136.500 MHz for broadcast of FIS weather products. The aircraft’s data link radio 
must be tuned to one of the two frequencies to receive weather information from the ap- 
propriate provider. 

Weather information will be broadcast from each ground station at established intervals. 
Upon full deployment, each FIS provider will provide coverage throughout the National 
Airspace System (NAS). 
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Appendix B. Aeronautical Information Manual FISDL Guidance 

Issue: 2/24/00 


7-1-10. FLIGHT INFORMATION SERVICES DATA LINK (FISDL) 

a. FISDL. Aeronautical weather and operational information may be displayed in the 
cockpit through the use of FISDL. FISDL systems are comprised of two basic types: 
broadcast systems and two-way systems. Broadcast system components include a terres- 
trial or pace-based transmitter, an aircraft receiver, and a cockpit display device. Two- 
way systems utilize transmitter/receivers at both the terrestrial or space-based site and the 
aircraft. 


1. Broadcast FISDL allows the pilot to passively collect weather and operational 
data and to call up that data for review at the appropriate time. In addition to text weather 
products, such as METAR's and TAF's, graphical weather products, such as radar com- 
posite/mosaic images may be provided to the cockpit. Two-way FISDL services permit 
the pilot to make specific weather and operational information requests for cockpit dis- 
play. 


2. FISDL services are available from three types of service providers. 

(a) Through vendors operating under a service agreement with the FA A 
using broadcast data link on VHF aeronautical spectrum (products and 
services are defined under subparagraph c). 

(b) Through vendors operating under customer contract on aeronautical 
spectrum. 

(c) Through vendors operating under customer contract on other than 
aeronautical spectrum. 


3. FISDL is a method of disseminating aeronautical weather and operational data 
which augments pilot voice communication with Flight Service Stations (FSS's), other 
Air Traffic Control (ATC) facilities or Airline Operations Control Centers (AOCC's). 
FISDL does not replace pilot and controller/flight service specialist/aircraft dispatcher 
voice communication for critical weather or operational information interpretation. 
FISDL, however, can provide the background information that can abbreviate and greatly 
improve the usefulness of such communications. As such, FISDL serves to enhance pilot 
situational awareness and improve safety. 
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b. Operational Use of FISDL. Regardless of the type of FISDL system being used, ei- 
ther under FAA service agreement or by an independent provider, several factors must be 
considered when using FISDL. 

1. Before using FISDL in flight operations, pilots and other flight crew members 
should become completely familiar with the operation of the FISDL system to be used, 
airborne equipment to be used, including system architecture, airborne system compo- 
nents, service volume and other limitations of the particular system, modes of operation 
and the indications of various system failures. Users should also be familiar with the con- 
tent and format of the services available from the FISDL provider(s). Sources of informa- 
tion that may provide this guidance include manufacturer's manuals, training programs 
and reference guides. 

2. FISDL does not serve as the sole source of aeronautical weather and opera- 
tional information. ATC, FSS, and, if applicable, AOCC VHF/HF voice is the basic 
method of communicating aeronautical weather, special use airspace, NOT AM and other 
operational information to aircraft in flight. FISDL augments ATC/FSS/AOCC services, 
and, in some applications, offers the advantage of graphical data. By using FISDL for 
orientation, the usefulness of any information received from conventional voice sources 
may be greatly enhanced. FISDL may alert the pilot to specific areas of concern, which 
will more accurately focus requests made to FSS or AOCC for inflight briefings or que- 
ries made to ATC. 

3. The aeronautical environment is constantly changing; often these changes oc- 
cur quickly, and without warning. It is important that critical decisions be based on the 
most timely and appropriate data available. Consequently, when differences exist be- 
tween FISDL and information obtained by voice communication with ATC, FSS, and/or 
AOCC (if applicable), pilots are cautioned to use the most recent data from the most au- 
thoritative source. 

4. FISDL products, such as ground-based radar precipitation maps, are not appro- 
priate for use in tactical severe weather avoidance, such as negotiating a path through a 
weather hazard area (an area where a pilot cannot reliably divert around hazardous 
weather, such as a broken line of thunderstorms). FISDL supports strategic weather deci- 
sion making such as route selection to avoid a weather hazard area in its entirety. The 
misuse of information beyond its applicability may place the pilot and his/her aircraft in 
great jeopardy. In addition, FISDL should never be used in lieu of an individual pre-flight 
weather and flight planning briefing. 

5. FISDL supports better pilot decision making by increasing situational aware- 
ness. The best decision making is based on using information from a variety of sources. 

In addition to FISDL, pilots should take advantage of other weather/NAS status sources, 
including, but not limited to, Flight Service Stations, Flight Watch, other air traffic con- 
trol facilities, airline operation control centers, pilot reports, and their own personal ob- 
servations. 


84 



c. FAA FISDL. The FAA's FISDL system provides flight crews of properly equipped 
aircraft with a cockpit display of certain aeronautical weather and flight operational in- 
formation. This information is displayed using both text and graphic format. This system 
is scheduled for initial operational capability (IOC) in the first quarter of calendar year 
2000. The system is operated by vendors under a service agreement with the FAA, using 
broadcast data link on aeronautical spectrum on four 25 kHz spaced frequencies from 
136.425 through 136.500 MHz. FISDL is designed to provide coverage throughout the 
continental U.S. from 5,000 feet AGL to 17,500 feet MSL, except in those areas where 
this is unfeasible due to mountainous terrain. Aircraft operating near transmitter sites will 
receive useable FISDL signals at altitudes lower than 5000 feet AGL, including on the 
surface in some locations, depending on transmitter/aircraft line of sight geometry. Air- 
craft operating above 17,500 MSL may also receive useable FISDL signals under certain 
circumstances. 

1. FAA FISDL provides, free of charge, the following basic products: 

(a) Aviation Routine Weather Reports (METAR's). 

(b) Special Aviation Reports (SPECI's). 

(c) Terminal Area Forecasts (TAF's), and their amendments. 

(d) Significant Meteorological Information (SIGMET's). 

(e) Convective SIGMET's. 

(f) Airman's Meteorological Information (AIRMET's). 

(g) Pilot Reports (both urgent and routine) (PIREP's); and, 

(h) Severe Weather Forecast Alerts (AWW's) issued by the FAA or NWS. 

2 . The format and coding of these products are described in Advisory Circular 
AC-00-45, Aviation Weather Services, and paragraph 7-1-28. Key to Aviation Routine 
Weather Report (METAR) and Aerodrome Forecasts (TAF). 

3 . Additional products, called Value-Added Products, are available from the ven- 
dors on a paid subscription basis. Details concerning the content, format, symbology and 
cost of these products may be obtained from the following vendors: 

(a) BENDIX/KING WxSIGHT 

Allied Signal, Inc. 

One Technology Center 

23500 West 105th Street 

Olathe, KS 66061 

(913)712-2613 

www. bendixking. com 
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(b) ARNAV Systems, Inc. 

16923 Meridian East 
P. O. Box 73730 
Puyallup, WA 98373 
(253) 848-6060 
www. amav. com 

d. Non-FAA FISDL Systems. In addition to FAA FISDL, several commercial vendors 
provide customers with FISDF on both the aeronautical spectrum and other frequencies 
using a variety of data link protocols. In some cases, the vendors provide only the com- 
munications system which carries customer messages, such as the Aircraft Communica- 
tions Addressing and Reporting System (ACARS) used by many air carrier and other op- 
erators. 


1. Operators using non-FAA FISDF for inflight weather and operational informa- 
tion should ensure that the products used conform to the FAA/NWS standards. Specifi- 
cally, aviation weather information should meet the following criteria: 

(a) The products should be either FAA/NWS accepted aviation weather 
reports or products, or based on FAA/NWS accepted aviation weather reports or 
products. If products are used which do not meet this criteria, they should be so 
identified. The operator must determine the applicability of such products to flight 
operations. 

(b) In the case of a weather product which is the result of the application 
of a process which alters the form, function or content of the base FAA/NWS ac- 
cepted weather product(s), that process, and any limitations to the application of 
the resultant product should be described in the vendor's user guidance material. 

2. An example would be a NEXRAD radar composite/mosaic map, which has 
been modified by changing the scaling resolution. The methodology of assigning reflec- 
tivity values to the resultant image components should be described in the vendor's guid- 
ance material to ensure that the user can accurately interpret the displayed data. 

3. To ensure airman compliance with Federal Aviation Regulations, National Airspace 
System (NAS) status products (such as NOTAM's, Special Use Airspace Status, etc.) and 
other government flight information should include verbatim transmissions of FAA prod- 
ucts. If these products are modified, the modification process, and any limitations of the 
resultant product should be described in the vendor's user guidance. 
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Appendix C. The Risk Assessment Task (RAT) 

(Index of Risk Taking Predilection) 

In the previous AWIN experiment, pilots were allocated to experimental groups in part 
based on their scores on a risk assessment task (RAT). This appendix briefly describes 
the history, pedigree, and rationale for employing the risk assessment task. 

Rationale 

Behaviors such as the dangerous misuse of weather displays need only take place once 
every few hundred hours of flight to have a significant adverse impact on flight safety 
and the incident/accident statistics. However, such a behavior is unlikely to be spotted in 
an hour or two of flight simulation, even if other realistic features of the operational envi- 
ronment are faithfully reproduced. Purely random sampling of the pilot population, there- 
fore, may not expose potentially unsafe behavior. 

The challenge, therefore, has been to utilize expert knowledge of flying, accident causa- 
tion and aviation psychology to recreate the kind of environment and circumstances that 
could be expected to increase the probability of detecting misuse of the new weather dis- 
play technologies. The experiment is not designed to calculate the prevalence of these 
“misuse behaviors”, but rather to evaluate and assess whether and how such misuse could 
occur, thus providing guidance for pilots and display manufacturers. 

With this in mind the team elected to proceed with a stratified random sample of pilots 
taken from populations that might be identified as higher risk and lower risk pilots. If 
weather display misuse accidents are going to occur, then pilots low in weather knowl- 
edge sophistication, high in risk acceptance, and motivated to continue a flight seem 
likely to be over-represented in the incident/accident statistics. Therefore, the subjects 
were pre-screened for weather knowledge and risk aversion. The purpose of the RAT task 
is to increase the probability of including subject pilots who might exhibit behaviors that 
would otherwise only emerge in the operational environment. The RAT task is not adver- 
tised as a definitive biographical variable or as a definitive measure or predictor of pilots’ 
decision-making prowess. The RAT does, however, have a well -documented history as a 
psychometric instrument, and, indeed, in a range of applied psychological studies includ- 
ing aviation, as outlined below. 

Given the risk construct used in the task, the research evidence to date, and the absence of 
alternative screening methods for our purpose, use of the RAT is a rational and low-risk 
option. The subject experiment is not designed to address questions relating to the rela- 
tionship between risk scores and decisions made in simulated flight. The experiment 
seeks to reproduce and characterize misuses of the weather display in an operational en- 
vironment. The absence of significant relationships observed between risk, as measured 
by the RAT, and other elements in the experiment, will not adversely impact the experi- 
mental outcome. 
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Origins of the RAT 

The RAT task is one sub-task from a version of a multiple task computerized battery of 
cognitive tasks that was explicitly designed to evaluate aviators (Banich, Stokes & El- 
ledge, 1989). The original research began with an information-processing task analysis of 
aviation and initially identified six primary areas of aviator cognitive proficiency that the 
battery should cover: working memory, attention (divided and focussed), spatial ability, 
logical reasoning, perceptual-motor abilities, and processing flexibility (or prioritizing). 
Risk taking predilection or aversion, that is, risk judgment, was subsequently added, as 
this was clearly a source of pilot variance not captured under the original six headings. 

Task sensitivity testing, reporting, and reliability 

All of the subtasks were tested against each other empirically in a series of discriminatory 
analyses (Stokes, et al., 1991a). The sensitivity, specificity and positive predictive value 
for each subtask in all seven areas of cognitive proficiency were compared, and thus an 
objective basis for comparison in standard epidemiological terms was determined. Stokes 
(1999) showed that the tests in the battery are reliable, do not suffer from undue practice 
effects, and factor load on the appropriate constructs. The risk task is particularly strong 
in these respects, exhibiting no practice effect and, whereas certain tasks (e.g. maze trac- 
ing, hidden figure recognition and spatial memory) all factor load onto one construct (e.g. 
spatial ability), only the RAT factor loaded on the risk construct. 

Applications of the RAT 

The original and updated versions of the battery have been utilized extensively in a range 
of applied studies. (Stokes et al., 1991b, 1994, 1995, 1997). Several studies have resulted 
in findings that involve the risk construct. 

For example, risk assessment appears to be a specific ability or cognitive dimension that 
can be directly impaired by neurological deficit. In a clinical study, (Stokes et. al., 1991a) 
showed that pilots’ evidenced less propensity for risk taking than members of a group of 
subjects did with known neuropsychological diagnoses (Stokes, 1991a, p.785). Moreover, 
the range of conditions in these diagnoses was broad, including as it did cerebro-vascular 
conditions, trauma, neuro degenerative disease and sequelae of alcoholism. 

An effect often observed and commented on in the engineering psychology literature 
(see, for example, Wickens, 1992), is the apparent conservatism associated with age. The 
clinical study also scrutinized the extent to which older subjects become more risk averse. 
Generally speaking, risk taking did indeed decrease with age in pilots, while it increased 
with age in the clinical group. The mechanism underlying the latter, clinical finding is not 
well understood, but the RAT findings for pilots are consistent with the wider literature 
on risk and risk aversion, increasing confidence in the utility of the measure. 

The RAT task has also been used in a series of double-blind clinical studies of the cogni- 
tive effects of aspartame (in the artificial sweetener ‘NutraSweef ) and of alcohol upon 
pilots. Aspartame was not found to affect cognition in either acute or chronic dosing, but 
alcohol, as anticipated, did. However, a number of new effects of alcohol were identified. 
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including an increased variability in risk taking. Mean RAT scores were the same in the 
alcohol and non-alcohol conditions, but this conceals a significantly greater variance 
around the mean in the alcohol condition. Scrutiny of the data showed that this was not a 
group level effect, but indeed did arise from greater capriciousness in trial to trial re- 
sponses of individuals as they worked through the RAT. Reproduced in the operational 
environment such swings from conservative, risk averse responding to high risk gambling 
could be expected to have a negative impact on safety. 

The RAT and Flight Training 

In a 1995 study, the effect upon flight training success of a number of information proc- 
essing variables, including risk predilection, was examined in the context of university 
flight training (Stokes & Bohan, 1995b). This study also evaluated the predictive utility 
of anxiety scores and academic grades. A major influence upon the outcomes of such 
studies is the nature of the criterion of success. In the 1995 study, several criteria were 
examined, including a checkride score, hours to solo, landings to solo, and ground school 
grade. The first three of these are closely associated with psychomotor skill, as the crite- 
ria involve maneuvering flight, rather than primarily cognitive skills such as those in- 
volved in cross-country flight management. The criteria used are reflected in the results - 
dual-task tracking tasks best predict success where maneuvering flight is the criterion. An 
unanticipated finding was that the risk task was predictive of ground school performance 
(which presumably includes a more “cerebral” element and little psychomotor control). 
The effect, however, was weak. In this study, important additional criteria were exam- 
ined. Results were compiled for checkride “passers” and “failers”, as one might expect, 
but also for individuals who had not been permitted to take the checkride. 

The significance of this may not be immediately obvious. It is necessary to know that in- 
structors were required to “sign off’ a student as being ready for the checkride. More- 
over, the sign off required that the student fly solo prior to the checkride. Understandably, 
instructors do not wish to be the agent of someone’s demise, and will not permit those at 
risk to fly solo. Therefore, they are dropped from the flightcheck pool. 

Given this, it can be argued that the real dichotomy is not between checkride “passers” 
and “failers.” A bigger performance gap presumably exists between those students signed 
off and those not signed off for the checkride, than between persons passing and failing 
the ride (all of whom had been adjudged fit to fly unaccompanied). In this light, the RAT 
scores were revisited. In fact, the highest risk scores were seen among the “not recom- 
mended” group (significantly higher than “failers”). “Passers” and “failers” did not differ 
significantly on the risk dimension. A compelling explanation for these results (and one 
supported by instructor comment) is that during flight training instructors had observed, 
among other defects such as poor psychomotor control, unsafe (“risky”) behaviors in cer- 
tain students and had declined to sign them off for solo or for the checkride. Although 
unknown to the instructors, these students indeed did have elevated risk task scores in the 
battery administered three months earlier, before the student had commenced flight train- 
ing at all. 
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Appendix D. Weather Knowledge Questionnaire and Key 


The weather knowledge test presented here is the key used to grade the test. All graded 
answers are shown in gray highlight. All other questions were not graded, as they were 
not relevant to weather knowledge and were just used as distracters. 
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General Aviation Questionnaire 

Thank you for participating in our Research Triangle Institute/NASA/FAA evaluation of 
advanced aviation technologies. We would like to learn a little more about your aviation 
knowledge before you participate in our study. Please take a couple of minutes to answer 
a few questions. Your answers are strictly confidential and will not be released. 

Name: Date: 

Phone number: E-Mail: 

1 . How many years have you been a pilot? 

2. What is your level of pilot certification (circle one)? 

Recreational Private Commercial Airline Transport 

3. What is your approximate number of total flight hours? 

4. Are you an instrument rated pilot? 

If so, are you current to fly instruments? 

5. What does a narrow temperature/dewpoint spread mean? 

Possible Fog 

6. How many feet are there in a statute mile? 

7. What does RVR stand for, and what does it mean? 

Runway Visual Range, Visibility down Specific Runway 

8. What COMM frequency can you use to contact Flight Watch 9 122.2. 122,0 

9. Briefly, describe class C and class G airspace. 

Class C: 

Class G: 


92 



10. How much does 20 gallons of 100 LL fuel weigh? 


1 1. What instrument indications would you notice, on take-off, if the static ports were 
blocked? 


12. What are the altitude limits of class A airspace, and what flight rules apply when fly- 
ing in that airspace? 


13. If you are flying eastbound, and you have a tailwind, would you typically be north or 
south of a low-pressure zone? 

South 


14. On a surface analysis weather chart, what do closely spaced isobars mean? 

High Winds 

15. What are you likely to see on the instruments if a pitot tube becomes blocked during 
the enroute phase of flight? Describe each phase. 

Level (accelerating): 

Climb: 

Descent: 


16. In what weather products can you find icing information? 

PIREPs. SIGMETs. TAFs. AIRMETs. Area Forecasts. Prognosis Charts. Compos- 
ite Moisture Charts. Wind Aloft Tables 

17. What do boundary layer air, and surface winds near the ground have in common? 

Both are slower than surrounding air, due to friction of the surface. 
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18. On a weather chart, what do the following symbols stand for? 



Occluded Front 



19. What type of information is found in an FDC NOTAM? 

Regulatory Notices. Charting Changes 

20. If a thunderstorm is identified as being severe, or giving an intense radar echo, what 
does the AIM say about how far you should avoid the storm? 

20 miles 

21. What do the following METAR/TAF weather codes stand for? 


RA = Rain 

SO = Sauall 

BR = Mist 

FZ = Freezing 

FC = Tornado 

DZ = Drizzle 

SH = Showers 

FU = Smoke 

FG = Fog 

GR = Hail 

SN = Snow 

1C = Ice Crystals 

HZ = Haze 

TS = Thunderstorm 


22. What is a void time clearance? 


94 


23. On a radar summary chart, what does the notation “NA” mean? 


Not available 


24. During a night time IFR flight, what clues suggest airframe icing? 


25. Please translate the following METAR weather report: 

METAR KDCA 291554Z 26012G18KT 10SM SCT040 BKN100 

15/05 A2985 


Regan National, 29 th day, 1 554 Zulu time, wind 260 degrees at 12 knots - gust- 
ing to 1 8 knots, visibility 10 statute miles, scattered clouds at 4000 feet, broken 
clouds at 10,000 feet, temperature 15 degrees Celsius, dewpoint 5 degrees Cel- 
sius, altimeter 29.85 inches of Mercury. 


Thank you for taking the time to complete our questionnaire, we appreciate your help. If 
we select you for our simulator study of advanced technologies, we will contact you by 
phone or E-mail. 
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Appendix E. Pre-Flight Weather Briefing 


Pre-Flight Weather Briefing 

As part of the mission preflight briefing materials, each pilot was given a paper copy of a 
standard weather briefing that would have been received by a call to a Flight Service Sta- 
tion telephone briefer. Both the teletype coded reports were given as well as an English 
translation. 


Adverse Conditions: 


AIRMET (WA) TANGO FOR TURB VALID UNTIL 272100Z 

AIRMET TURB...MD VA NC 

FROM EMI TO SBY TO RDU TO PSK TO EMI 

AFT 18Z OCNL MOD TURB BLW 060 DUE TO INCRG SWLY FLOW AHD OF 
CDFNT. CONDS SPRDG EWD AND CONTG BYD 21Z THRU 03Z. 


AIRMET (WA) TANGO for turbulence valid until twenty-one hundred universal coordi- 
nated time for Maryland, Virginia, and North Carolina. 

From Westminster (EMI), Virginia to Salisbury (SBY), Maryland, to Raleigh-Durham 
(RDU), North Carolina to Pulaski (PSK), Virginia to Westminster (EMI), Virginia. 

After one, eight, zero, zero, universal coordinated time, occasional moderate turbulence 
below six thousand feet due to increasing southwesterly flow ahead of cold front. Condi- 
tions spreading eastward and continuing beyond twenty-one hundred universal coordi- 
nated time, and through zero, three, zero, zero universal coordinated time. 


Synopsis: 

At one, seven, zero, zero universal coordinate time, a Cold Front extending from south- 
west Pennsylvania along the Appalachians through Central West Virginia, Western Vir- 
ginia, and Eastern Tennessee, northwest Georgia and Central Alabama will continue to 
move Eastward. 

A warm front extending from southwest Pennsylvania Eastward to Atlantic City, NJ. will 
continue to move Northeastward, and a Trough of Low Pressure extending from north- 
west West Virginia southward into Central South Carolina will continue moving East- 
ward. 
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Current Conditions: 


PHF SA 1800Z M 8 BKN 07 14/12/0910/992 

Newport News, Williamsburg International Airport weather report, one, eight, zero, zero 
universal coordinated time. Measured ceiling eight hundred broken, visibility seven, tem- 
perature one, four, dew point one, two, wind zero niner, zero at ten, altimeter two, niner, 
niner, two. 

RIC SA 1800Z 50 SCT M70 BKN 05 14/12/3010/992 

Richmond International Airport weather report, one, eight, zero, zero universal coordi- 
nated time. Five thousand scattered, measured ceiling seven thousand broken, visibility 5, 
temperature one, four, dew point one, two, wind three, zero, zero at one zero, altimeter 
two, niner, niner, two. 


OFP SA 1747Z E50 BKN 150 OVC 10 18/12/2015/960 

Richmond, Hanover County Airport weather report, one, seven, four, seven universal co- 
ordinated time. Estimated ceiling five thousand broken, one, five thousand overcast, visi- 
bility one, zero, temperature one, eight, dew point one, two, wind two, zero, zero, at one, 
five, altimeter two, niner, six, zero. 

LKU SA 1750Z 20 SCT E40 BKN 100 OVC 10 17/12/2415G20/955 

Louisa County, Freeman Airport weather report, one, seven, five, zero universal coordi- 
nated time. Two thousand scattered, estimated ceiling four thousand broken, one, zero 
thousand overcast, visibility one, zero, temperature one, seven, dew point one, two, wind 
two, four, zero at one, five gusting two, zero, altimeter two, niner, five, five. 

WAL SA 1749Z CLR BLO 120 10 16/10/1810/969 

NASA, Wallops Airport weather report, one, seven, four, niner universal coordinated 
time. Clear of clouds below one, two thousand, visibility one, zero, temperature one, six, 
dew point one, zero, wind one, eight, zero at one, zero, altimeter two, niner, six, niner. 

MFV SA 1753Z CLR BLO 120 10 18/11/1806/968 

Accomack County Airport, Virginia weather report one, seven, five, three universal co- 
ordinated time. Clear of clouds below one, two thousand, visibility one, zero, temperature 
one, eight, dew point one, one, wind one, eight, zero at six, altimeter two, niner, six, 
eight. 

SBY SA 1750Z CLR BLO 120 10 16/10/1810/969 

Salisbury, Maryland weather report one, seven, five, zero universal coordinated time. 
Clear of clouds below one, two thousand, visibility one, zero, temperature one, six, dew 
point one, zero, wind one, eight zero at one, zero, altimeter two, niner, six, niner. 
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UA: /OV RIC150015 /TM 1720Z /FL 040/TP C180 /SK SCT150 /TB LGT 

Pilot report one-five miles southeast of Richmond, Virginia. At one, seven, two, zero 
universal coordinated time. At four thousand feet, a Cessna one, eighty reported in clouds 
with light turbulence. 

UA: /OV SBY /TM 1715Z /FL 030 /TP MO20 /SK SCT150 /TB NEG 

Pilot report over Salisbury, Maryland at one, seven, one, five universal coordinated time. 
At three thousand feet, a Mooney reported clouds at one five thousand scattered, and 
negative turbulence. 


UA: /OV RIC045025 /TM 1710Z /FL 040 /TP C172 /TB LGT-MOD 

Pilot report two-five miles northeast of Richmond, Virginia at one, seven, one, zero uni- 
versal coordinated time. At four thousand feet, a Cessna one, seven, two reported light to 
moderate turbulence. 

Satellite Imagery indicates several Cumulus clouds beginning to develop throughout cen- 
tral Virginia, including the Richmond area, over the past hour. 

Weather Radar at one, seven, one, zero universal coordinated time indicates scattered ar- 
eas of light to moderate rain showers in Central Virginia, but no precipitation in the East- 
ern sections of the state. 


En-Route Forecast: 


TAF KPHF 271729Z 271818 16014G24KT P6SM SCT100 BKN200 BECMG 2022 
16017G27KT SCT060 OVC120 

FM0000 1618G25KT P6SM SCT030 OVC060 TEMPO 5SM -SHRA OVC030 
PROB40 0103 VRB20G40KT 2SM TSRA OVC020 CB 

Terminal area forecast for Newport News-Williamsburg International Airport. Valid from 
one seven, two nine, to one eight, one eight, universal coordinated time, wind one, six, 
zero at one, four gusting two, four, visibility unrestricted, scattered clouds at one, zero 
thousand, broken clouds at two, zero thousand. Conditions becoming between two, zero, 
zero universal coordinated time and two, two, zero, zero universal coordinated time, wind 
one, six, zero at one, seven gusting two, seven, scattered clouds at six thousand, overcast 
at one, two thousand until zero, zero, zero, zero universal coordinated time. 
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Central and Eastern Virginia Area Forecast: 


271800Z SCT-BKN050 OVC120 TOP 200, OTLK VFR TSRA 

The area forecast for Central and Eastern Virginia after one, eight, zero, zero universal 
coordinated time: scattered to broken clouds at five thousand, overcast at one, two thou- 
sand, tops at two, zero thousand, outlook VFR with thunderstorms and rain. 

TAF KRIC 271729Z 271818 18018G20KT P6SM SCT060 OVC120 BCMG2022 
OCNL -SHRA OVC030 PROB40 2302 VRB20G40KT 2SM TSRA OVC020CB 

Terminal area forecast for Richmond International Airport. Valid from one seven, two 
nine, to one eight, one eight, universal coordinated time, wind one, eight, zero at one, 
eight gusting two, zero, visibility unrestricted, scattered clouds at six thousand, overcast 
at one, two thousand. Conditions becoming between two, zero, zero, zero universal coor- 
dinated time and two, two, zero, zero universal coordinated time, occasional light rain 
showers, overcast at three thousand, with a chance of thunderstorms after two, three, 
zero, zero universal coordinated time. 

TAF KSBY 271729Z 18018 1820G30KT SCT100 BKN200 

Terminal area forecast for Salisbury, Maryland after one seven, two nine, universal coor- 
dinated time. Wind one, eight, zero at two, zero gusting three, zero, scattered clouds at 
one, zero thousand, broken clouds at two zero thousand, visibility unrestricted. 

Winds Aloft Forecast: 

030 060 090 

ORF 1920 2025+5 2130+2 
RIC 2020 2125+4 2130+1 

Winds aloft forecast for the Norfolk, and Richmond, Virginia areas after one, seven, zero, 
zero universal coordinated time.Norfolk at three thousand: wind one, niner, zero at two, 
zero. At six thousand: wind two, zero, zero at two, five, temperature plus five. At niner 
thousand: wind two, one, zero at three, zero, temperature plus two. Richmond at three 
thousand: wind two, zero, zero at two, zero. At six thousand: wind two, one zero at two, 
five, temperature plus four. At niner thousand: wind two, one, zero at three, zero, tem- 
perature plus one. 

NOTAMS: 

No Current NOTAMS Fisted. 

ATC Delays: 

NONE 

ATC request PIREPS for turbulence or other conditions along your route of flight. Con- 
tact Flight Watch or Flight Service. Washington Flight Watch is available with En-route 
Flight Advisory Service to update your weather briefing on 122.0MHz. Feesburg Flight 
Service station is available on 122.2 MHz for weather briefings and other in-flight ser- 
vices. 
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Appendix F. Cockpit Research Facility Description 


The RTI/NASA Cockpit Research Facility (CRF) was configured for the experiment as a 
conventionally equipped aircraft with the addition of a display of FIS-B information. The 
CRF consists of three major subsystems (as illustrated in Figure F-l): 

• Rapid Prototype Simulator Cab - Consists of the cockpit mockup with controls, in- 
struments, radios and indicators. A Closed Circuit Television (CCTV) camera is 
mounted behind and above the pilots’ left shoulder to provide live images from the 
cockpit to the Scenario Controller and Observer Position. 

• Scenario Controller and Observer Position - Consists of the master control station, 
which is used for scenario generation, selection, monitoring and recording of flight 
progress. Provides the operator and experiment observer with displays of all control 
positions, radio and instrument switch positions, instrument displays and the Out-the- 
Window (OTW) (as presented to the subject pilot). A weather data display of 
NEXRAD images is provided for the scenario controller and for the observer to track 
the flight’s progress relative to the weather. A video image of the cockpit from the 
CCTV camera is provided for the observer to monitor the subject pilot’s actions. Live 
audio of all radio transmissions between the pilot and the NAS (controller, Flight 
Watch, ATIS, etc.) are available to the scenario controller and the observer. An inter- 
com audio network is provided which allows private conversations between the sce- 
nario controller, observer and air traffic controller positions. Simulated radio trans- 
missions between the pilot and air traffic controller are also enabled over the same in- 
tercom system. All intercom traffic is recorded on the audio track that accompanies 
the video recording made from the CCTV camera. 

• ATC Controller Position - Consists of a custom ATC station developed for perform- 
ing experiments of this type and a weather display that shows the latest NEXRAD 
images to track the flight’s progress relative to the weather. Current pilot-selected 
COM frequencies are displayed so that the ATC controller can verify that the pilot is 
contacting ATC on the correct frequency before responding to an initial contact. 
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Figure F-1. Cockpit Research Facility 
Simulator Cab Description 

The simulator cab is a two-seat cockpit mockup designed for single-pilot IFR operations. 
The basic ergonomic structure of the mock-up is patterned after a generic GA airplane in 
terms of the relative placement and types of controls and instruments, instrument panel 
width and height, and seat placement. The pilot’s position is outfitted with complete con- 
trols including a yoke, rudder pedals, instruments, switches and indicators as described in 
the paragraphs below. 

The center console holds the radios and throttle quadrant. Both cockpit positions are out- 
fitted with headsets and an intercom system that allows the pilot to communicate with a 
passenger and with simulated Air Traffic Control. A 37-inch monitor mounted directly in 
front of the pilot, approximately at the position of the aircraft nose provides the primary 
out-the- window view. 

Controls, Instruments and Indicators Description 

The controls, instruments and indicator configurations available are typical of those 
found in an IFR-equipped aircraft as shown in Figure F-2. 

All instrument panel round dial indicators are rendered on flat panel liquid crystal dis- 
plays (LCDs). The 14-inch diagonal LCDs provide enough display area to fully render 
the standard instrument “T” configuration with other supplemental indicators as well. A 
second 14-inch diagonal LCD provides display area for navigation instruments and en- 
gine parameter indicators. Table F-1 lists the types of instruments rendered in the cockpit. 
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All instruments provide the operational performance required by the Federal Aviation 
Administration Federal Aviation Regulations, Society of Automotive Engineers Aero- 
space Standards and RTCA, Inc. performance specifications as applicable to simulation. 



Mags & 
Master 



RH HRRH 


7 777 

Electric Trim Switch 
Gear Switch 
Flap Switch 

Gear Indicators 
Throttle Quadrant 


Switches 

1 Taxi Lights 
Landing Lights 
Rotating Beacon 
NAV Lights 
Fuel Pump 
Pitot Heat 


Trim Indicator 


Figure F-2. Instrument Panel, Controls and Indicators 
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Table F-1. Instruments and Indicators in the instrument Panel 


Flight Control 
Instruments 

Navigation / 
Communications 
Instruments 

Engine Monitoring 
Instruments 

Indicators 

• Attitude 

• Airspeed 

• Altitude 

• Turn Coordinator 

• Compass Slaved 
Directional Gyro 

• Vertical Speed 

• VOR / DME 
Display #1 with 
ILS Localizer 
& Glideslope 

• VOR / DME 
Display #2 with 
ILS Localizer 
& Glideslope 

• NAV Radio #1 

• NAV Radio #2 

• COM Radio #1 

• ADF 

• Transponder 

• Manifold Pres- 
sure (MAP) 

• Engine RPM 

• Fuel Quantity 

• Fuel Flow 

• Oil Pressure 

• Exhaust Gas 
Temperature 
(EGT) 

• Cylinder Head 
Temperature 
(CHT) 

• Trim Position 

• Flap Position 

• Marker Beacon 

• Gear Position 

• Autopilot Mode 


The control yoke provides the pilot with an electric trim button, push-to-talk switch for 
the intercom system, chronometer for time calculations as well as a full range of control 
movement for controlling the flight path of the airplane. Activation of the electric trim 
button moves the yoke in or out to relieve the control forces. The current trim position is 
displayed on a trim indicator in the instrument panel. A display indicator on the PFD no- 
tifies the pilot that the autopilot is engaged and if it needs additional trimming. 

Visual System and Displays Description 

A Silicon Graphics Onyx 10000 is used to generate the instrument panel gages and the 
Out-the-Window scene for the pilot. A rapid prototyping tool is used to develop and ren- 
der the regulation-compliant gages in appearance and performance. Round dial instru- 
ments are rendered on two 14-inch diagonal active matrix Liquid Crystal Displays 
(AMLCDs), each having an addressable resolution of 1024 pixels by 768 lines. 

The OTW scene is a photo-textured presentation rendered at a 40 degree horizontal by 30 
degree vertical field-of-view and displayed on a 37-inch monitor at an addressable resolu- 
tion of 1280 pixels by 1024 lines. The monitor is positioned so that active display area 
subtends approximately 40 degrees horizontal to the pilot’s eye point. Both instrument 
displays and OTW scene are rendered at a 30 Hz frame rate with a 70 Hz display refresh 
rate. 

A visual terrain database for the state of Virginia contains six major airports at which 
takeoffs and landings can be made. Another 24 airports are rendered at photographic 
quality to facilitate pilotage along several routes between NASA Langley, Newport 
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News/Williamsburg, Blacksburg, Richmond, Manassas, Washington National and NASA 
Wallops Island runways. The environmental conditions be varied to achieve any meteoro- 
logical conditions required, i.e. overcast, lowRVR, cloud decks, etc. 

Data Acquisition System Description 

The data acquisition system is used to collect information about the pilot’s control inputs 
and switch actions, format the data, and transfer the data to the SGI Onyx for processing 
in the simulation models. 

A data acquisition controller system is hosted in a Pentium 60-based PC. The data acqui- 
sition controller contains a microcontroller that performs all input / output (I/O) opera- 
tions with the hardware in the simulator cab. Operations performed by the controller in- 
clude: 

• Acquiring analog control position information 

• Performing the analog to digital conversions on control position information 

• Acquiring switch position discretes 

• Driving indicators in the cab (e.g. Gear Position Indicators, Outer Marker , 
Middle Marker) 

• Acquiring frequency selections set in the COM, NAV 1, NAV 2, ADF, Trans- 
ponder and autopilot interfaces in the radio stack located in the cockpit center 
console 

• Acquiring the OBS 1 & 2, HDG select and baro Altimeter knob settings 

• Updating the frequency displays in the COM, NAV 1, NAV 2 and ADF radios 


Simulation Control, Monitoring and Recording 

The simulation control and flight path monitoring process running in the Silicon Graphics 
Indigo controls all aspects of the simulation. The simulation control process initializes the 
simulation models in the SGI Onyx, performs real time data display and data collection 
capture of various flight parameters for later analysis, and presents a plan view of the air- 
craft’s position during operation of the simulation, similar to an ATC console. The sys- 
tem operator uses the simulation control to select various scenarios, position / reposition 
aircraft model and monitor scenario progress. 

Table F-2 lists the real-time parameters displayed at the operator’s station during system 
operation. Table F-3 lists the data dictionary of parameters available for collection and 
reduction. 
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Table F-2. Real-time Parameters Displayed During Operations 


Parameter 

Parameter 

Parameter 

Parameter 

Airspeed (A/S) 

• Calculated A/S 

• Indicated A/S 

• True A/S 

Aerodynamic Coeffi- 
cients 

• CL Total Lift 

• CD Total Drag 

• C Y Total Side Force 

• CM Total Pitching Moment 

• CR Total Rolling Moment 

• CN Total Yawing Moment 

Control Surface De- 
flection 

• Elevator 

• Rudder 

• Aileron 

• Aileron Trim 

• Rudder Trim 

• Trailing Edge Flaps 

Ground Contact Con- 
ditions (Landing) 

• Rate of Decent 

• Bank Angle 

• Side & Vertical 
Forces on Nose 
Gear 

• Side & Vertical 
Forces on Left Gear 

• Side & Vertical 
Forces on Right 
Gear 

• Ground Speed 

Altitude 

• Pressure 

• AGL 

Position 

• Latitude 

• Longitude 

• Pleading 


Aircraft Body 
Angles 

• Pitch 

• Roll 

• Yaw 

• Angle of Attack a 

• Sideslip (3 

Atmospheric 

• OAT 

• Air Pressure 

• Baro Pressure 

Weight & Balance 

• Gross Weight 

• Payload 

• Total Fuel 

• CG relative to 35% 
Mean Aerodynamic 
Chord (MAC) 


• Z-Load (# Gs 
through the polar 
axis) 

• Engine Thrust 

• Rate of Climb 
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Table F-3. Dictionary of Recordable Parameters and Inducible Faults 


Parameters 

Aerodynamic Model 

Altitude 

Altitude Pressure 

Aileron Position 

Column Force 

Elevator Position 

Elevator Trim position 

Rudder Position 

Stall Buffet 

Wheel Force 

Indicated Airspeed 

Altimeter Setting 

Indicated Rate of Climb 

Roll Attitude 

On Ground Status 

Weight on Gear 


Weight on Left Gear 

Weight on Right Gear 

Angle of Attack 

Side Slip Angle 

Pitch Angle 

Pitch Acceleration 

Actual Rate of Climb 

Roll Angle 

Calculated Airspeed 

Ground Speed 

True Airspeed 


| Atmosphere | 

| Atmospheric Pressure 

] OAT degrees C 

] OAT degrees F 

Ambient Air Pressure j 

Autopilot - 22 Parameters 

Circuit Breakers - 20 Breakers 

ICE - Induced, Pitot Head, etc - 19 Parameters 

Gear - True Gear Positions, Nose, Left, Right 

1 NAV | 

ADF Indicator 

DME Distance 

DME Speed 

DME Time 

DME Mode Switch 

OBS 1, OBS 2 

CDI 1 , CDI 2 

Glide Slope 1 , Glide 
Slope 2 

Magnetic Heading 

Outer Marker 

Middle Marker 


| Induced Faults | 

Runaway trim 

Autopilot Pitch Axis 
Failure 

Autopilot Hard Roll 

Autopilot Soft Roll 

Autopilot Circuit 
Breaker 

Dead Battery 

Fuel Pump Failure 

NAV 1 Failure 

Nav 2 Failure 


Vacuum 

System 

Pos 

tion 

Standby Vacuum On 

Vacuum Hg 

Latitude 

Longitude 

Vacuum Enunciator 

Pump Switch 

| Coni 

trols | 

| Throttle Position 

] Prop Position 

] Mixture Position 

1 \ 

| Weight and Balance | 

Center of Gravity 

Long Load Force 

CG % MAC 

Total Weight 

Passenger & Baggage 
Weight 

Fuel Weight 




Air Traffic Management Console 

The Air Traffic Management (ATM) console is used during the conduct of research pro- 
jects to provide a more realistic environment for the subject pilot involved in the re- 
search. During investigations, an experienced Air Traffic Controller operates the station. 
The ATM station receives data from the simulator and presents it on the ATM Station 
monitor in a manner sufficient to support the ATM functions required of the Air Traffic 
Controller. 

The screen consists of “radar image data” and associated mapping features, Figure F-3. 
System controls and informational data are presented on the side and top of the display. 
The operator may zoom in to a 1-mile scale (used for ground control) to a 100-mile scale 
(approach, departure, and enroute functions). Features that can be displayed during the 
operation of the ATM Station include: intersections (with and without names), airports 
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(with and without names), runways (utilized during approaches), taxiways (utilized for 
ground control), VORs (with and without names), and special use airspace. The display is 
centered upon the selected airport (currently PHF, RIC, LFI, or WAL). Future implemen- 
tations include the display and manipulation of the flight paths of multiple aircraft. 



Figure F-3. Air Traffic Management Console Display 


Aerodynamic Simulation Model 

The simulation model is a 6 Degree of Freedom (DOF) aerodynamic model that is table 
driven to provide the performance characteristics for the Piper Malibu PA46-310P. The 
Piper Malibu represents a high performance single engine GA with a cruising speed of 
170 knots. The simulation model is executed in the SGI Onyx, based on the pilot inputs 
collected through the data acquisition system. It is computed in 3 parts: fast rate (30 
Hertz) coefficients, medium rate (15 Hertz) coefficients and slow rate coefficients (7.5 
Hertz). 

The aerodynamic coefficients in the simulation model incorporate the non-linear charac- 
teristics of an operational airplane. These adjustments give the simulation model more 
realistic longitudinal handling characteristics and make it possible, for example, to flare 
the airplane to a maximum lift stall at the touchdown point if desired. 
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Crosswind Model 

A crosswind model is available that can direct a cross wind over a large range from any 
direction. 

Turbulence Model 

A basic turbulence weather model is included in the simulation. As the aircraft ap- 
proaches a weather system, the level of turbulence can be increased based on the overall 
level of convective weather and distance from the weather. 

Four levels of turbulence are calculated, from mild (level 1) to heavy (level 4). For each 
level, a random turbulence factor is added into the wind velocity for each of the respec- 
tive axis wind velocities. All instruments react to the turbulence in a manner reflecting 
the movement of the airframe through the airmass, i.e. rapid fluctuations in airspeed, ver- 
tical speed, attitude, heading, etc. 

The autopilot is programmed to disengage when the aircraft flies into areas of level three 
turbulence or higher. Attempts to reengage the autopilot while in this level of turbulence 
will result in an automatic disengagement within 10 seconds. 
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Appendix G. NEXRAD Mosaic Images - 4km Cells 



Figure G-1. 1900Z NEXRAD Mosaic Image — 4 km Cells 



INEXRAD 03/27 19:07; 


SmiN 


NEX/MTR 


Menu Off 


Figure G-2. 1907Z NEXRAD Mosaic Image — 4 km Cells 
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Figure G-10. 2003Z NEXRAD Mosaic Image — 4 km Cells 
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Appendix H. NEXRAD Mosaic Images - 8km Cells 



Figure H-1. 1900Z NEXRAD Mosaic Image — 8 km Cells 



Figure H-2. 1907Z NEXRAD Mosaic Image — 8 km Cells 
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Figure H-3. 1914Z NEXRAD Mosaic Image — 8 km Cells 



hXflAU 113/2 / 19:21Z Vj 


NEX/MTR 


Figure H-4. 1921Z NEXRAD Mosaic Image — 8 km Cells 
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Figure H-5. 1928Z NEXRAD Mosaic Image — 8 km Cells 



Figure H-6. 1935Z NEXRAD Mosaic Image — 8 km Cells 
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Figure H-7. 1942Z NEXRAD Mosaic Image — 8 km Cells 



Figure H-8. 1949Z NEXRAD Mosaic Image — 8 km Cells 
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Figure H-9. 1956Z NEXRAD Mosaic Image — 8 km Cells 


Figure H-10. 2003Z NEXRAD Mosaic Image — 8 km Cells 
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Appendix I. Experiment Briefing 


This appendix shows the briefing given to the pilots upon first entering the simulation 
facility. It explains the overall structure, schedule, equipment and mission of the experi- 
ment. An explanation of the weather display is also given at this point. 


FAA/NASA/RTI 
Flight Information Services 
Data Link (FISDL) 
Experiment 

June, 2000 
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Experiment Procedure 



Pre-Test Phase Test Phase 



Subject Pilot Schedule 

Time 

Activity 

0:20 

Introduction 

1:30 

Simulator Familiarization 

0:10 

Break 

0:30 

Flight Planning 

1:30 

Flight Experiment 

0:30 

Debriefing 
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Today’s Flight Mission 


Situation 

• A diabetic patient is in urgent need of insulin at 
Wallops Island on eastern shore of Virginia. 

• The insulin is vital to survival of the patient. The 
longer the delay, the greater the likelihood that 
patient will not survive, or at best, suffer serious 
eomplieations 

•Potentially fatal complications include Diabatic 
Ketoacidosis (DKA). One therapy for DKA 
includes treatment with sodium bicarbonate. 

Today’s Mission /R I 

• RTI Medical Services, Inc. is to deliver insulin 
to the Wallops Island airport from NNWB 
airport, stopping enroute to pick up sodium 
bicarbonate at Richmond, Va airport. 

(The sodium bicarbonate medicine will be driven 
out to A/C at end of runway) 

• Departure of the RTI Medical Services flight is 
1900 hours this evening. 
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Simulation Hardware Configuration 



Observer 

Position 




Simulation Cockpit Configuration 


Weather Information Sources 

• ATIS 

• Flight Service Station 

• Flight Watch 

• Virginia AWOS/ASOS Reports via radio 

• Air Traffic Control (IAW normal NAS procedures) 

Tower 

Departure 

Enroute 

Approach 

• Data Link Flight Information Services (FISDL) Display 
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; 


Zoom 


Graphics 


Menu Off 


50 mile scale, graphics only mode 
just showing airports in gray boxes 
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Modes 


Display Mode: 
Graphics 
METAR 
NEX/METAR 


Movement Mode 
Zoom 
Scroll 
Crosshair 


Joystick 


Softkeys 


Mom 


Menu Display 
Mode: 


GPS Mode 
Off 
On 
Free 


Brightness 


25 mile scale, graphics only mode. 
Showing airports in gray boxes, and 
NavAids in blue circles (navaids only 
shown in scales of 25 miles or less) 
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Scroll 


METAR 


Menu Off 


10 mile scale, METAR mode. 
Showing airports in gray boxes, 
NavAids in blue circles, and graphical 
METARs 
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N EXRAD 05/25 12:572 


Scroll 


NEX/WTR; 


NEXRAD Intensity Levels: 

Lt. Green WM Light 
Dk. Green ■ 

Yellow □ 

Dk. Yellow □ 

Magenta ■ Severe 


25 mile scale, NEXRAD/METAR mode. 
Showing airports in gray boxes, 

NavAids in blue circles, graphical 

METARs and NEXRAD image with time stamp. 
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Appendix J. Simulator Briefing and Training 


All the pilots were provided the opportunity to complete a practice flight in the simulator. 
The researcher/trainer guided the subject pilot through maneuvers to acquaint them with 
the operation and performance of the simulator. Additional instruction was given to the 
pilots on the weather display. 

Weather Display Experiment Simulator Familiarization Flight Syllabus 

1. General explanation of cockpit layout: 

Primary flight instruments 

Secondary instruments 

Sub panel controls and systems 

Yoke controls 

Radios, Autopilot, Intercom 

Charts 

2. Checklist explanation 

3. Engine start and taxi 

4. Run-up and system check 

5. Normal takeoff and climb 

6. Level off at 3000 feet (± 100 feet) 

7. Shallow and steep banked turns to a heading (± 1 0 degrees) 

8. Autopilot: 

Engage/ di sengage 
Pitch modifier 
Altitude hold 
Altitude modifier 
Heading hold 

9. VOR operation (on AP) 

10. Use of weather display (on AP) 

11. Vectors to normal VFR landing, touch-and-go (± 10 kts) 

12. Go-around (± 10 kts) 

13. Vectors to IFR approach and landing (second landing if required) 
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Appendix K. Observer Form 


Pilot Name: Subject# 

Condition: Ownship Only/Large Cells — Risk Score: WX Score: 


Richmond Decision 


Actions 

Weather Information 

Take Off: 

Radio Inquiry (times) 

Divert to Wallops: time plate 

Newport News Departure: 

Position: 

Flight Service: 

Hold at: 

Fliqht Watch: 

Time Beqan: 

Richmond Approach: 

Time End: 


Then: 

Richmond Tower: 


Other: 

Commence Richmond Approach 


OM time: 

Automated Services (times) 

Self Abort: 


Waved Off: 

Newport News ATIS: 


Richmond ATIS: 

Abort Mission 

Other: 

Time: 


Place: 

Weather Display 

Distance to Red Cell at Break: 

Textual METAR: 


Station: 

Other Action: 

Time: 


Other: 

Al 

1 fliqht data confirmed: initial 
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Wallops Decision 


Actions 

Given Weather Warning at: 

Penetrated Storms: Time 

Position: 

Hold at: 

Time Began: 

Time End: 

Then: 

Diverted Around Storms: 

North: 

South: 

Closest Distance to Red Cell: _ 

Abort Mission 

Time: 

Place: 

Other Action 


Weather Information 

Radio Inquiry (times) 

Richmond Departure: 

Flight Service: 

Flight Watch: 

Norfolk Approach: 

PAX River Approach: 

Wallops Tower: 

Other: 

Automated Services (times) 

Richmond ATIS: 

Wallops ASOS: 

Other: 

Weather Display 

Textual METAR: 

Station: 

Time: 

Other: 


All flight data confirmed: initial 
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Appendix L. Immediate Reactions Questionnaire 


IMMEDIATE REACTIONS QUESTIONNAIRE 

Please use the following scales to rate how much you agree or disagree with the 
statements offered. If some items appear to overlap, do not be concerned, but at- 
tempt to answer each on its own terms. There are no “right” or “wrong” answers, 
nor any agenda of preferred responses being sought by the researchers. 


1. I took the medical emergency scenario seriously, in the sense that I factored the emer- 
gency into my decision making. 

Disagree Disagree No Opinion Agree Agree 

Somewhat Somewhat 


2. An advantage of the onboard Weather Display was showing the weather in real-time, 
that is, as it actually was at that moment? 

Disagree Disagree No Opinion Agree Agree 

Somewhat Somewhat 


3. I attribute much of my decision making to my interpretation of the Weather Display. 

Disagree Disagree No Opinion Agree Agree 

Somewhat Somewhat 


4. 1 tried to systematically sample all sources of weather information open to me. 

Disagree Disagree No Opinion Agree Agree 

Somewhat Somewhat 


5. I used the weather display, but felt the need to cross-check or verify my conclusions 
from conventional weather data sources (ATC, etc.). 

Disagree Disagree No Opinion Agree Agree 

Somewhat Somewhat 
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6. I felt comfortable with the autopilot, in terms of understanding its use and operation. 


Disagree 


Disagree No Opinion Agree Agree 

Somewhat Somewhat 


7. Without the autopilot, my completion of the flight would have been compromised. 

Disagree Disagree No Opinion Agree Agree 

Somewhat Somewhat 


8. The degree of validity of the weather data appearing on the display was a factor I felt 
that I held in mind as I flew. 

Disagree Disagree No Opinion Agree Agree 

Somewhat Somewhat 


9. I have been monitoring the weather display time-stamp regularly in my instrument 
scan. 

Disagree Disagree No Opinion Agree Agree 

Somewhat Somewhat 


10. At the time of my arrival to the Richmond airport, I knew that there was a storm — 
(circle one) 

a. about 10 nm northwest of the airport 

b. about 5 nm northwest of the airport 

c. near the airport 

d. right at the airport 


11. At the time I was en-route to Wallops Island, I saw across my path of direct flight, 
what I took to be — 

a. a penetrable storm 

b. a navigable opening between convective cells 

c. a non-navigable opening between cells 

d. a wall of convective activity requiring diversion. 
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12. On the weather display, I found the positional accuracy of the aircraft icon to be ade- 
quate. 

Disagree Disagree No Opinion Agree Agree 

Somewhat Somewhat 


13. In using the weather display, I felt that I generally knew the aircraft position relative 
to any storms. 

Disagree Disagree No Opinion Agree Agree 

Somewhat Somewhat 


14. 1 felt that I had adequate sources of weather information to make confident decisions. 


Disagree 


Disagree No Opinion 
Somewhat 


Agree Agree 

Somewhat 


Thank you for your help. 
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Appendix M. Structured Interview Guide 

Richmond Decision Interview 

The following questions refer to only the leg between Newport News and Richmond. 


Decision Rationale 

1. What led you to make the decision to: 


2. What information did you use to make that decision? 


3. Do you feel that you had enough information to make a sound and confident judge- 
ment of the situation? 


4. What were your primary and secondary sources of weather information? 

Primary: 

Secondary: 


5. Was there any information that you lacked and would have liked to have? 


6a. Was there information that was available, but that you didn’t use? 


6b. Why didn’t you obtain that information? 
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7. Do you feel that you had enough time to gather all the information that you wanted? 


8. Did you ever consider holding? 


Wallops Decision Interview 

The following questions refer to only the leg between Richmond and Wallops Island. 


Decision Rationale 

9. What led you to make the decision to: 


10. What information did you use to make that decision? 


1 1. Do you feel that you had enough information to make a sound and confident judge- 
ment of the situation? 


12. What were your primary and secondary sources of weather information? 

Primary: 

Secondary: 


13. Was there any information that you lacked and would have liked to have? 
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14a. Was there information that was available, but that you didn’t use? 


14b. Why didn’t you obtain that information? 


15. Do you feel that you had enough time to gather all the information that you wanted? 


The following questions apply to the entire flight. 


Weather Interpretation 

17a. How close are you willing to fly near hazardous weather conditions, and what do 
you consider hazardous conditions? 


18. How close do you think that you flew to a red cell? 

Richmond — 

Wallops — 


Routing 

19. Did ATC help or hinder your route planning? 

20. At any time, did you consider totally aborting the mission? 

21. If you had a chance to do the flight again, would you do anything different? 
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Weather Display 


22. How would you compare the weather display information to other sources of weather 
information (ATIS, Flight Watch, etc.)? 


23. Did you use the weather display to help you navigate? 


24. Do you feel that the weather display increased or decreased your workload? 


25. Did you trust the weather display to give you correct information? 


26. Was there any confusion with the ownship symbology? 


27. How did you determine your distance to the weather cells? 


Comfort Zone 

28a. Did you feel ahead of the airplane? 


28b. If not, at what times did you feel behind the airplane, and what was the biggest con- 
tributor? 
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Appendix N. Weather Display Questionnaire 


Subject Pilot #: Date: 

1 . Approximately how many total hours do you have? 

2. Approximately how many actual instrument hours do you have? 

3. Approximately how many hours do you have under the hood in flight? 

4. Approximately how many hours do you have in a simulator ? 

5. Approximately how many instrument hours do you have in the last 90 days ? 

6. What ratings do you have? (circle as many as apply) 

Private Commercial ATP Glider Airship Sea 

Instrument CFI CFII MEI Heli copter A&P I A 

7. What type of aircraft do you have most of your experience in? 


8. Have you ever used a datalinked inflight weather display system in a flight? 
(not including onboard radar or Stormscope) 

If yes, how many flights do you have with it? 


9. If you answered yes to the last question, how many times have you used that data- 
linked inflight weather display system to make actual weather judgements? (Instead of 
just experimenting with the display). 


10. Have you had any training in weather interpretation other than basic pilot training 
(for example, courses in meteorology)? If so, what? 
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11. What is your usual method of obtaining a pre-flight weather briefing? 
(DUATS, FSS phone, etc.) 


12. Have you tried other alternate methods of weather briefings, and what was 
your experience? 


13. In using this weather display today, did you find the operation straightforward? 
If not, what operations of this weather display did you find difficult? 


14. In using this weather display today, did you find the graphical METAR symbology 
useful? If not, what features did you find difficult? 


15. In using this weather display today, did you find the textual METAR presentation 
useful? If not, what features did you find difficult? 


16. Considering your use of the weather display today, would you like to see any 
additional features or change any existing features? 


17. How did the weather display affect your decisions that you made today? 


18. Do you feel that you needed more training on the weather display? 
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If so, in what areas? 


19. How did you determine the age of the weather information? 


20. Were there any features about the weather display that caused you to cast doubt as to 
its usefulness in normal, real world, operation? 


21 . Did you find that the weather display increased or decreased your workload? 


22. How did the use of the autopilot help or hinder the use of the weather display? 


23. Did you find that the weather display increased or decreased your 

situational awareness? 


Thank you very much for participating in our study, we appreciate your help. 
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Appendix O. Air Traffic Control Scripts 

Communication Exchanges Between Pilot and ATC 

The following is a typical communication exchange for the mission. Each pilot deviated 
from this typical exchange, some more than others, but only to the extent of clarifying 
radio calls, routing changes and exchanges to gather weather information. 


FIRST LEG 


SUMMARY: INSTRUMENT FLIGHT FROM NEWPORT NEWS/ 
WILLIAMSBURG (PHF) AIRPORT TO RICHMOND INTERNATIONAL (RIC) 
AIRPORT VIA DIRECT HOPEWELL V260 RICHMOND. SEVERE 
THUNDERSTORM APPROACHING RIC. PILOT TO DECIDE WHETHER TO 
CONTINUE APPROACH AND ATTEMPT LANDING AT RIC, HOLD 
AWAITING WEATHER IMPROVEMENT, OR BY-PASS RIC AND REQUEST 
CLEARANCE TO WALLOPS, VA (WAL) FLIGHT FACILITY. 


N73Y: (Tunes 128.65 for ATIS) 

ATIS: THIS IS NEWPORT NEWS WILLIAMSBURG INTERNATIONAL 
TOWER INFORMATION BRAVO. 1800 ZULU MEASURED CEILING 1000 
OVERCAST VISIBILITY 3 MILES. TEMPERATURE14 DEWPOINT 12 WIND 
090 AT 10 ALTIMETER 29.92. LANDING AND DEPARTING RUNWAY 7. ILS 
RUNWAY 7 APPROACH IN USE. ADVISE YOU HAVE BRAVO. 


N73Y: Newport News clearance delivery, Malibu 2573Y ready for clearance. (121.65) 

ATC: MALIBU 2573Y CLEARED TO RICHMOND VOR VIA DIRECT 
HOPEWELL V260 RICHMOND MAINTAIN 5000. SQUAWK 1424. 

N73Y: Roger, cleared to Richmond via direct Hopewell V260 Richmond maintain 5000. 

(Tunes 121.9) 

N73Y: Newport News ground control, N73Y ready to taxi, have information Bravo. 


ATC: N73Y, GROUND CONTROL, TAXI STRAIGHT AHEAD THEN LEFT TO 
RUNWAY 7. WHEN READY FOR TAKEOFF, CONTACT TOWER ON 118.7. 


N73Y: Malibu 73Y, Roger. 
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(Tunes 118.7) 

N73Y: Tower, N73Y ready for takeoff. 


ATC: N73Y MAINTAIN RUNWAY HEADING FOR RADAR VECTORS 
HOPEWELL MAINTAIN 2000, EXPECT CLEARANCE TO 5000 WITHIN 5 
MINUTES AFTER DEPARTURE. CLEARED FOR TAKEOFF RUNWAY 7. 


N73Y: Malibu 73 Y Roger, cleared for takeoff. 


(Departs) 

ATC: MALIBU 73Y CONTACT NORFOLK DEPARTURE CONTROL ON 124.9. 


(Tunes 124.9) 

N73 Y : Norfolk departure control, this is N73 Y climbing to 2000 on runway heading. 


ATC: N73Y ROGER, IN RADAR CONTACT. TURN LEFT PROCEED DIRECT 
HOPEWELL, CLIMB AND MAINTAIN 5000 


N73Y: Malibu 73 Y Roger, Proceeding direct Hopewell. 


N73Y: Norfolk departure control, request permission to leave frequency for Richmond 
ATIS. 

ATC: N73Y FREQUENCY CHANGE APPROVED. ADVISE WHEN BACK ON 
MY FREQUENCY. 

(N73Y tunes 1 19. 15 for RIC ATIS) 

ATIS: THIS IS RICHMOND TOWER INFORMATION DELTA. 1910 ZULU 
MEASURED CEILING 200 OVERCAST VISIBILITY THREE QUARTERS 
THUNDERSTORMS MODERATE RAIN SHOWERS TEMPERATURE 14 
DEWPOINT 12 WIND 300 AT 10 ALTIMETER 29.92. ILS RUNWAY 34 
APPROACH IN USE. LANDING AND DEPARTING ON RUNWAY 34. ADVISE 
YOU HAVE DELTA. 


(Tunes 124.9) 


N73Y: Departure control, N73Y back on your frequency. 

ATC: MALIBU 73Y ROGER. 

ATC: MALIBU 73Y CONTACT RICHMOND APPROACH CONTROL ON 134.7. 
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(Tunes 134.7) 

N73Y: Richmond approach control, this is Malibu 73 Y. Have information Delta. 


ATC: N73Y, RICHMOND APPROACH CONTROL, ROGER, DESCEND AND 
MAINTAIN 2000. EXPECT VECTORS TO ILS RUNWAY 34 APPROACH. 


N73Y: N73Y, Roger, descending to 2000. 

ATC: N73Y DEPART HOPEWELL VOR HEADING 300 FOR A VECTOR TO 
ILS RUNWAY 34 FINAL APPROACH COURSE. 


N73Y: N73Y, Roger, depart Hopewell heading 300 for vector to ILS runway 34 ap- 
proach course. 

ATC: MALIBU 73Y, 4 MILES SOUTHEAST OF KAFKA, MAINTAIN 2000 
UNTIL ESTABLISHED ON THE LOCALIZER, CLEARED FOR ILS RUNWAY 
34 APPROACH. CONTACT TOWER ON 121.1 PASSING KAFKA. 

N73Y: Malibu 73Y, Roger, cleared for approach, tower 121.1 at KAFKA. 


(Tunes 121.1) 

ATC BROADCAST: ATTENTION ALL AIRCRAFT IN RICHMOND AREA. 
LOW LEVEL WINDSHEAR ADVISORIES IN EFFECT FOR RICHMOND 
INTERNATIONAL AIRPORT. 

(ATC TO IMPROVISE HOLDING, CLEARANCE TO WALLOPS, OR MISSED 
APPROACH DEPENDING ON PILOTS DECISION/REQUEST WITH 
WEATHER ENCOUNTERED.) 
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2ND LEG 


SUMMARY: INSTRUMENT FLIGHT FROM RICHMOND (RIC) TO WALLOPS 
FLIGHT FACILITY (WAL) VIA RICHMOND DIRECT HARCUM DIRECT 
JAMIE VI MAGGO. AT PILOT’S REQUEST AFTER HOLDING OR 
EXECUTING A MISSED APPROACH AT RICHMOND. 

ATC: MALIBU 73Y CLEARED TO THE MAGGO INTERSECTION VIA 
DIRECT HARCUM DIRECT JAMIE VI MAGGO. CLIMB AND MAINTAIN 
5000 CONTACT RICHMOND DEPARTURE CONTROL ON 126.4. 


N73Y: Malibu 73Y, Roger, proceeding direct Harcum climbing to 5000, changing to 
126.4. 


(Tunes 126.4) 

N73Y: Richmond departure control Malibu 73Y proceeding direct Harcum climbing to 
5000. 

ATC: N73Y, ROGER, IN RADAR CONTACT. 


N73Y: Departure control, N73Y, request permission to leave frequency for Wallops 
ASOS information. 


ATC: N73Y FREQUENCY CHANGE APPROVED. ADVISE WHEN BACK ON 
MY FREQUENCY. 


(Tunes 119.175) 

WALLOPS AUTOMATED SURFACE OBSERVATION WIND 170 AT 6 
VISIBILITY 3 MILES. MEASURED CEILING 1000 BROKEN TEMPERATURE 23 
DEWPOINT 16 AL TIMETER 29. 92. 


(Tunes 126.4) 

N73Y: Richmond departure control, Malibu 73 Y back on your frequency. 

ATC: MALIBU 73 Y, ROGER. 

ATC: (When N73Y is approximately 10-15 nm from weather cells depicted.) N73Y, I 
SHOW WEA THER AHEAD. AD VISE INTENTIONS. 

(Possible requests from N73Y as weather is encountered.) 
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N73Y: 1) Request deviation to south/north to avoid weather. 

2) What do you show for weather on my route of flight? 

3) Request vector around weather. 

4) Request a new route/altitude to avoid weather. 

5) Request frequency change for Flight Watch or FSS. 


ATC: RESPOND TO SPECIFIC REQUEST, I. E: 

1) UNABLE TO APPROVE DEVIA TION TO THE NORTH. RESTRICTED 
AREA 6609 IN USE. 

2) DEVIATION TO THE SOUTH APPROVED. 

3) I SHOW HEA VY WEA THER ON YOUR PROJECTED FLIGHT PA TH. 

4) ROGER, TURN RIGHT HEADING FORA VECTOR SOUTH OF 

WEATHER. 

5) FREQUENCY CHANGE APPROVED. ADVISE WHEN BACK ON MY 
FREQUENCY. 


ATC: N73Y CLEAR OF WEATHER FLY HEADING FOR VECTOR TO VI. 

N73Y: N73Y, Roger , turning to heading 

ATC: N73Y CONTACT PATUXENT APPROACH CONTROL ON 127.95. 


N73Y: N73Y Roger changing to 127.95 
(Tunes to 127.95) 

N73Y: Patuxent approach control, this is Malibu 73 Y. 


ATC: N73Y THIS IS PATUXENT APPROACH CONTROL, EXPECT VOR/DME 
RUNWAY 10 APPROACH TO WALLOPS. ALTIMETER 29.92. 


N73Y: N73Y, Roger. 

ATC: N73Y, DESCEND AND MAINTAIN 2000. 
N73Y: N73Y, Roger, leaving 5000 for 2000. 


ATC: (5 Miles south ofMAGGO) TURN RIGHT HEADING 060 INTERCEPT THE 
SALISBURY 24. 1 MILE ARC CLEARED FOR VOR/DME RUNWA Y 10 
APPROACH. 
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N73Y: N73Y, Roger, heading 060 to the arc, cleared for VORDME Runway 10 ap- 
proach. 

ATC: MALIBU 73Y CONTACT WALLOPS TOWER ON 126.5. 

N73Y: Malibu 73Y, Roger changing to tower. 

(Tunes 126.5) 

N73Y: Wallops Tower, this is Malibu 73Y on approach to runway 10. 

ATC: MALIBU 73Y, WALLOPS TOWER, WIND 170 AT 6, CLEARED TO LAND 
RUNWAY 10. 
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Appendix P. Enroute Weather Report Scripts 

The following weather report script was available to the Air Traffic Controller to be used 
as updated weather information. The reports were available while the mission was in pro- 
gress, but the information was only given to the pilot if requested. These reports were 
available through the Flight Service Station radio, Enroute Flight Advisory Service 
(Flight Watch) and Air Traffic Control frequencies. Also included in this Appendix are 
the scripts used in the pre-recorded weather broadcasts of: Newport News ATIS, Rich- 
mond ATIS and Wallops Island ASOS. 

En-route Abbreviated Weather Reports 

AIRMET (WA) TANGO FOR OCNL MOD TURB BLO 060 for MD, VA and NC is 
current. 

ZDC CWA01 1855Z Valid Until 2100Z 
FROM CSN TO RIC TO DAN TO LYH TO CSN 

BKN AREA OF TSRA INCRG IN INTENSITY AND COVERAGE MOV EAST 

Washington Center Weather Advisory zero, one valid until two, one, zero, zero universal 
coordinated time. From Casanova, Virginia to Richmond, Virginia, to Danville, Virginia, 
to Lynchburg, Virginia, to Casanova, Virginia. Broken area of thunderstorms and rain 
increasing in intensity and coverage, moving east. 

ZDC CWA02 1855Z VALID UNTIL 2100Z 
FROM SBY225025 TO RIC090050 

BKN LINE OF TSRA INCRG IN INTENSITY AND COVERAGE MOV LITTLE 

Washington Center Weather Advisory zero, two valid until two, one, zero, zero universal 
coordinated time. From two, five miles Southwest of Salisbury, Maryland to Five, zero 
miles East of Richmond, Virginia. Broken line of thunderstorms and rain increasing in 
intensity and coverage, moving little. 

RIC SP 1910Z M002 OVC 3/4TRW 58/55/9012G16/992/TSTM OVHD 
OCNL LGTCCCG 

Richmond International Airport special weather report one, niner, one, zero universal co- 
ordinated time. Measured ceiling, two hundred, overcast, visibility %, thunderstorm, 
moderate ram showers, temperature five, eight, dew point five, five, wind zero, nine, 
zero, at one two, gusting one six, altimeter two nmer nine two, thunderstorm overhead, 
occasional lightning cloud to cloud, and cloud to ground. 
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LKU SP 1905Z MOOS OVC 1/2TRW+FG 57/57/2615G25/950/TSTM OVHD MOV E 
OCNL LGTCCCG 

Louisa County, Freeman Airport special weather report one, niner, zero, five universal 
coordinated time. Measured ceiling five hundred overcast, visibility one half, thunder- 
storm, heavy rain showers, fog, temperature five, seven, dew point five, seven, wind two, 
six, zero at one, five gusting two, five, altimeter two, niner, five, zero, thunderstorm 
overhead moving East, occasional lightning cloud to cloud, and cloud to ground. 

WAL SA 1846Z (Current) 

MVP SA 1846Z (Current) 

SBY SA 1845Z (Current) 

UA: /OV RIC /TM 1900Z /FL 010-SFC /TP C210 /TB SYR /RM LLWS FA 

Urgent pilot report over Richmond, Virginia at one, niner, zero, zero universal coordi- 
nated time. From one thousand feet to the surface, a Cessna two, one, zero reported se- 
vere turbulence and low-level wind shear on final approach. 

UA: /OV SBY /TM 1905Z /FL 060 /TP BE55 /TB NEG /RM MANY BLD-UPS OVR 
BAY SW 

Pilot report over Salisbury, Maryland at one, niner, zero, five universal coordinated time. 
At six thousand feet, a Beech five, five reported negative turbulence, and many build-ups 
over the bay Southwest. 

UA: /OV RIC090050 /TM 1900Z /FL080 /TP PA46 /TB NEG /RM BLD-UPS OVR 
BAYN 

Pilot report five, zero miles East of Richmond, Virginia at one, niner, zero, zero universal 
coordinated time. At eight thousand feet, a Piper four, six reported negative turbulence 
and build-ups over the bay North. 

Satellite Imagery indicates solid Build-Ups forming throughout Central Virginia. 

Weather Radar indicates solid light to moderate precipitation with increasing areas of 
Heavy precipitation developing throughout Central Virginia moving eastward into the 
Richmond (RIC), and Mecklenburg-Brunswick (AVC) areas. 
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Newport News ATIS 

this is Newport News-Williamsburg International tower information bravo, 
eighteen hundred zulu 
measured ceiling one thousand overcast 
visibility three miles 
temperature one-four 
dew point one-two 
wind, zero-niner-zero at one-zero 
altimeter, two-niner-niner-two 
landing and departing runway seven 
ILS rim way seven approach in use 
advise you have bravo 


Richmond ATIS 


this is Richmond tower information delta, nineteen-ten zulu 

measured ceiling two-hundred, overcast 

visibility three-quarters 

thunderstorms, moderate rain showers 

temperature one-four 

dewpoint one-two 

wind, three-zero-zero at one-zero 

altimeter, two-niner-mner-two 

ILS runway three-four approach in use 

landing and departing runway three-four 

advise you have delta 


Wallops Island ASPS 

Wallops automated surface observation 

wind, one-seven-zero at six 

visibility, three miles 

measured ceiling one-thousand, broken 

temperature, two-three 

dewpoint, one-six 

altimeter, two-niner-niner-two 
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